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Bureau  of  Naval  Personnel 
Washington,  D.  C.  20370 


The  purpose  of  this  report  was  to  evaluate  SHIP  II,  a  total  ship 
manpower  simulation  model  developed  under  advanced  development  funding. 
All  previous  research  apolications  were  first  reviewed.  The  data  were 
-then  used  to  evaluate  the  model  in  terms  of  a  comprehensive  set  of 
model  evaluation  criteria  whicn  included  validity  (predictive  and 
construct) ,  sensitivity,  reliability,  utility,  and  practicality.  The 
utility  of  SHIP  II  was  found  to  be  good;  SHIP  II  can  be  used  to  study 
many  types  of  manpower  problems  and,  for  some  problems,  can  provide 
data  not  available  from  other  sources.  The  practicality  of  SHIP  II 
is  low  mainly  because  SHIP  II  requires  a  large  amount  of  input  data 
and  is  costly  to  operate.  The  construct  validity  of  SHIP  II  was  found 
to  be  satisfactory.  The  data  was  not  sufficient  to  evaluate  the  model 
in  terms  of  predictive  validity,  reliability  and  sensitivity.  SHIP  II 
has  h-igh  enough  potential  utility  to  the  Navy  that  further  work  to  in¬ 
vestigate  the  validity,  sensitivity  and  reliability  of  the  model  would 
be  worthwhile,  especially  if  it  appears  that  SHIP  II  would  be  accepted 
by  Navy  manpower  planners. 
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SUMMARY 


Problem 


SHIP  II  is  a  computer  simulation  model  of  a  shipboard 
manpower  system  undergoing  advanced  development.  Although 
the  model  has  been  used  in  a  series  of  shipboard  manpower 
studies  which  have  produced  significant  findings,  Navy  man¬ 
power  planners  have  not  been  receptive  to  its  use  as  an 
operational  model  for  solving  shipboard  manning  problems. 

Such  reluctance  is  based  in  part  on  the  fact  that  data  on 
the  quality  and  utility  have  not  been  readily  available. 

This  report  will  provide  a  comprehensive  evaluation  of 
SHIP  II  quality  and  utility  together  with  discussion  of 
factors  affecting  the  future  use  of  the  model. 

Background  and  Requirements 

SHIP  II  has  been  supported  under  Advanced  Development 
funding  (TDP  P43  07X) .  The  funding  used  for  this  sub-project 
will  be  terminated  on  30  June  1973.  A  requirement  therefore 
exists  to  summarize  that  ADO  subproject  in  terms  of  the 
quality  and  utility  of  the  model  which  has  resulted  from  it. 

Approach 

The  evaluation  of  SHIP  II  was  performed  in  the  following 
steps: 


1.  A  set  of  model  evaluation  criteria  was  developed, 
based  on  the  model  evaluation  literature. 

2.  Each  study  using  SHIP  II  was  described. 

3.  The  data  from  each  study  was  evaluated  in  terms  of 
its  implications  for  each  of  the  evaluation  criteria. 

4.  The  study  results  were  summarized  to  obtain  judge¬ 
ments  of  the  overall  adequacy  of  SHIP  II  with  respect  to 
each  criterion. 

Findings 

1.  No  major  face  validity  problems  were  found,  but  the 
study  data  were  insufficient  to  provide  a  complete  evaluation 
of  SHIP  II  validity.  Face  validity  was  used  because  of  the 
absence  of  SHIP  II  predictive  validity  data. 
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2.  Construct  validity  of  the  model  is  satisfactory. 

3.  Sensitivity  is  adequate  except  possibly  for  two  input 
variables  studied.  But  data  for  only  part  of  the  input-output 
variable  combinations  of  SHIP  II  were  available  for  evaluation. 

4.  The  data  from  the  studies  were  not  sufficient  to 
evaluate  SHIP  II  reliability. 

5.  The  utility  of  SHIP  II  is  very  good.  SHIP  II  is 
appl.i cable  to  a  variety  of  Navy  manpower  problems. 

6.  The  practicality  of  SHIP  II  is  low  but  roughly  equal 
to  other  models  of  similar  si"  . 

Conclusions 


1.  SHIP  II  has  the  potential  to  provide  significant 
assistance  to  Navy  manpower  planners  which  is  not  available 
from  existing  methods.  The  types  of  problems  which  SHIP  II 
can  study  make  SHIP  II  a  potentially  valuable  addition  to 
Navy  manpower  study  methodology.  Few  other  models  similar 
to  SHIP  II  exist,  and  none  provide  the  versatility  which  is 
characteristic  of  SHIP  II. 

2.  The  potential  of  SHIP  II  cannot  be  fully  used 
because  the  technical  characteristics  of  the  model  are  still 
not  completely  known.  Of  the  five  major  criteria  used,  the 
data  for  only  two  and  part  of  .a  third  were  sufficient  to  pro¬ 
vide  an  adequate  evaluation  of  SHIP  II.  Hie  predictive 
validity,  reliability  and  sensitivity  of  SHIP  II  are  still 
largely  unknown. 

Recommendations 


1.  The  Navy  manpower  planning  community  should  be 

informed  about  how  SHIP  II  can  potentially  be  of  assistance 
in  the  study  of  ship  manpower  problems.  (p.  99-114)  * 

2.  Further  evaluation  studies  of  SHIP  II  would  be  worth¬ 
while  if  SHIP  II  would  receive  significant  recognition  by 
Navy  manpower  planners.  (p.  99-114) 

SKIP  II  should  be  transferred  to  the  newly  established 
Navy  rsonnel  Research  and  Development  Center  for  performance 
of  th_  additional  evaluation  studies.  (p.  99-114) 
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INTRODUCTION 


A.  Problem 


The  total  Ship  Simulation  Model  (Ship  n)  has  been  under  de¬ 
velopment  since  May  1969  and  has  been  successfully  applied  to 
two  classes  of  Navy  Destroyers.1  Since  Ship  n  has  the  po¬ 
tential  of  dealing  with  a  diversified  array  of  input  and 
output  variables  in  much  greater  depth  than  any  other 
known  model,  it  represents  a  valuable  addition  to  the  man¬ 
power  planning  process.  Notwithstanding  these  operational 
applications  of  the  model,  the  potential  usefullness  of 
the  model  has  not  been  fully  appreciated  and,  in  fact,  is 
not  well  known.  Manpower  planners  do  not  have  enough  in¬ 
formation  about  Ship  II  model  quality  and  uses  to  effectively 
apply  the  model  to  their  problems.  In  view  of  the  consider¬ 
able  resources  spent  to  develop  Ship  nf  a  critical  evaluation 
of  it's  utility  in  assisting  manpower  planners  is  necessary. 

The  results  of  this  evaluation  are  presented  in  this  report. 

B.  Objectives 

The  first  objective  of  chis  report  is  to  draw  together  all 
research  results  on  Ship  II  into  an  integrated  and  comprehen¬ 
sive  statement  of  the  quality  of  the  model.  The  information 
on  model  quality  is  intended  to  be  used  in  two  ways:  (1)  To 
evaluate  the  benefits  which  can  be  obtained  from  ship  II  in 
light  of  it's  development  costs,  (2)  To  assist  personnel  who 
have  manpower  problems  to  solve  and  want  to  determine  if 
SHIP  II  will  meet  their  needs. 

The  second  objective  is  to  present  enough  information  for 
a  user  to  determine  if  Ship  II  can  be  used  for  his  particular 
area  of  application.  This  type  of  information  includes 
mainly  detailed  descriptions  of  model  input  data  requirements 
and  output  data  types  and  formats.  This  information  is  not 
intended  to  be  sufficient  for  the  reader  to  understand  how 
the  model  operates;  the  intent  is  to  provide  enough  informa¬ 
tion  for  a  potential  user  to  determine  whether  the  use  of 
Ship  II  is  appropriate  and  feasible  for  his  application. 


1  The  two  classes  of  destroyers  to  which  Ship  II  has  been 
applied  are  the  DDG-2  and  the  DD  963.  Chapter  II  summar¬ 
izes  these  applications. 
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In  June  1967,  after  a  successful  application  of 
the  COSIMO  simulation  model  by  the  New  Developments  kosearch 
Branch  (1966)  ,  the  Bureau  of  Naval  Personnel  (Pers-A33)  ini¬ 
tiated  exploratory  development  research  (EDR)  concerned  with 
the  development  of  a  Total  Ship  Simulation  Model  (TSSM) .  The 
purpose  of  the  research  was  to  test  the  feasibility  of  deve¬ 
loping  a  simulation  model  of  an  entire  ship  so  that  personnel 
research  could  be  performed  at  the  total  system  level.  A 
simulation  advisory  group,  composed  of  representatives  of 
various  Navy  organizations  was  formed  to  provide  guidance  to 
the  effort. 


The  EDR  was  completed  in  June  1968  and  was  reviewed 
by  the  advisory  group.  The  results  of  the  review,  despite  a 
variety  of  cautions  concerning  cost,  scope  and  validity,  were 
generally  favorable  and  plans  were  made  to  continue  the 
research  into  the  advanced  development  phase.  These  plans 
provided  for  model  improvement,  data  refinement  and  the 
training  of  Navy  personnel  in  the  use  of  the  TSSM. 

By  April  1969,  following  refinement  of  the  data  base, 
the  first  analytic  study  (Spencer,  Lichtenberg,  Hass  and 
Byrnes,  1969)  was  performed.  Immediately  following  the  ap¬ 
plication,  the  model  was  reprogrammed  and  the  second 
generation  model  (Ship  II)  emerged.  Ship  II  has  since  been 
modified  and  improved  many  times. 

Ship  II  has  also  been  applied  by  NAVPERSRANDLAB  to 
several  types  of  problems  for  the  purpose  of  both  obtaining 
data  to  evaluate  the  model  and  to  assist  in  the  solution  of 
real  problems  for  Navy  manpower  planners. 


These  studies  using  Ship  II  have  been  conducted  under 
Advanced  Development  funding  (TDP  P43  07X).  The  purpose  of 
ADO  funding  was  to  test  the  feasibility  of  Ship  II  as  an 
operational  manpower  planning  tool.  Ship  II  has  been  under  de¬ 
velopment  for  a  sufficient  length  of  time  that  its  utility  and 
suitability  for  this  purpose  should  be  evaluated.  ADO  funding 
for  Ship  II  will  be  terminated  on  30  June  1973. 


Description  of  Ship  II 


a.  General  Description:  Ship  II  is  an  event- advance , 
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stochastic  simulation  model;  it  uses  manning,  equipment  and 
task  data  inputs;  generates  performance  times  from  distri¬ 
butions,  and  accounts  for  personnel  utilization,  task  perfor¬ 
mance,  and  equipment  readiness.  The  model  contains  an 
elaborate  personnel  assianment  logic  which  attempts  to  match 
personnel  to  jobs  or  watch  stations  with  a  minimum  of  queuing. 
This  is  done  by  employing,  when  necessary,  relief  watch- 
standers,  upward  absorption  of  work  (assigning  jobs  to  skill 
levels  higher  than  required  when  the  specified  skill  is  not 
available)  and/or  equivalent  NECS.  The  detailed  personnel 
record  together  with  the  flexible  assignment  logic  are  impor¬ 
tant  features  of  the  model.  During  a  simulation  run,  many 
of  the  functions  will  be  active  simultaneously;  in  some  cases 
conflicts  occur.  For  example,  if  the  employment  schedule 
calls  for  a  GQ  exercise,  all  hands  will  be  required  to  man 
stations  and,  therefore,  other  work  must  stop.  The  model 
handles  conflicts  such  as  this  by  allowing  jobs  to  be  inter¬ 
rupted  and  resumed  at  a  later  time  when  the  personnel  become 
available.  Throughout  the  simulation,  statistics  are  main¬ 
tained  on  each  type  of  activity.  Data  are  also  maintained  on 
the  readiness  of  the  ship  to  perform  mission  functions, 
including  equipment  readiness  and  training  readiness. 

Ship  II  is  basically  a  resource  allocation  model  in 
which  the  limited  resource  to  be  allocated  is  the  ships  crew 
and  it's  time.  There  are  many  different  types  of  demands 
which  are  placed  on  the  crew  which  conflict  in  time  require¬ 
ments,  priorities,  skill  requirements,  etc.  Basically, 

SHIP  II  is  a  device  for  keeping  track  of  the  resolution  of 
these  conflicting  demands  in  terms  of: 

(1)  Workload  for  each  crew  member  in  9  types  of 
activities 


etc. ) 


(2)  Task  completion  (maintenance  tasks,  ships  work. 


(3)  Equipment  state  and 

(4)  Ships  readiness 


Ship  II  incorporates  many  real  world  dynamic  inter¬ 
actions  which  cannot  be  included  in  analytic  models.  For 
example,  a  requirement  may  exist  for  a  job  to  be  done  and 
some  personnel  required  to  do  the  job  may  be  idle,  however, 
the  job  may  not  get  done  because  a  member  of  the  team  is  en¬ 
gaged  in  a  higher  priority  activity.  The  output  statistics 
would  then  show  a  job  queue  at  the  same  time  that  it  shows 


This  paragraph  is  taken  from  Hutchins,  Prather,  Barefoot 
and  Flint  (1973),  p.  A-475. 


personnel  idle. 

The  requirement  for  corrective  maintenance  occurs  when 
an  equipment  failure  is  found.  Equipment  failures  are 
generated  using  failure  rates  and  equipment  operating  times 
to  determine  when  the  malfunction  occurs.  When  a  malfunction 
is  found,  a  failure  is'  designateu  and  is  sent  to  the  trouble¬ 
shooting  function  for  fault  isolation  and  then  to  the  repair 
function  for  repair  and  verification.  Whenever  an  equipment 
failure  occurs  in  a  mission  oriented  equipment  item,  a 
readiness  decision  table  is  used  to  make  the  appropriate  al¬ 
teration  in  the  readiness  of  the  associated  subsystem  and  of 
the  total  ship. 

The  model  is  designed  so  that  every  individual  in  the 
crew  is  identified  separately  and  uniquely.  Each  man  is 
given  an  identification  number,  rating,  rate,  and  NEC,  when 
appropriate.  In  addition,  Ship  II  keeps  track  of  each  man 
so  that  his  status  is  always  known. 

The  model  incorporates  an  elaborate  personnel  assignment 
logic  which  matches  personnel  to  jobs  and  watch  stations* 

Some  of  the  techniques  used  to  obtain  men  for  assignments  are 
use  of  relief  watchstanders ,  assignment  of  jobs  to  skill 
levels  higher  than  required  when  to  specified  skill  is  not 
available,  and  using  equivalent  NEC’s.  Ship  II  follows 
certain  rules  in  job  assignment  which  can  be  varied  by  the 
user.  Given  these  rules,  Ship  II  then  assigns  men  dynamic¬ 
ally,  interrupting  lower  priority  jobs  to  assign  men  to 
higher  priority  jobs,  relieving  men  from  jobs  so  they  can 
stand  watch,  etc. 

During  a  simulation  run,  many  of  the ’shipboard  functions 
will  be  going  simultaneously.  If  conflicts  occur.  Ship  II 
handles  these  by  allowing  jobs  to  be  interrupted  by  higher 
priority  activities  and  resumed  at  a  later  time  as  personnel 
become  reavailable.  This  interrupt  function  occurs,  for 
example,  during  a  General  Quarters  exercise,  when  all  hands 
are  required  to  man  watch  stations. 

.  3 

b.  Basic  Model  Operation  ;  in  Ship  II  the  control 
and  processing  of  chronological  events  is  dependent  on  the 
use  of  a  "master  clock"  or  simply  "clock."  At  the  beginning, 
the  "clock"  is  set  to  zero  and  moved  ahead  depending  on  the 
occurrence  of  events  or  functions.  The  clock  has  no  real¬ 
time  meaning  in  that  a  simulation  which  lasts 


3  This  section  is  based  on  Smith,  Spencer  and  Brooks  (1970). 
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a  few  minutes  on  the  computer  may  have  simulated  many  days 
of  system  activity.  Because  the  program  controls  the  time 
of  simulation  via  the  "clock,"  it  is  also  possible  to 
simulate  simultaneous  events  merely  by  not  changing  time 
while  these  events  are  taking  place. 

Ship  II  is  an  event  advance  (Emshoff  and  Sisson,  1970) 
model.  In  this  type  of  model,  the  program  moves  the  master 
clock  ahead  just  far  enough  to  reach  the  next  event  and  then 

exercises  the  associated  logic.  Since  each  event  has  a 

time  associated  with  it,  the  program  need  only  check  the 
time  of  the  list  of  events  to  find  event  with  the  lowest 

time  value.  The  event  (or  events)  associated  with  this  mini¬ 

mum  time  is  called  the  imminent  event.  In  this  way  the 
program  moves  time  ahead  at  uneven  intervals.  In  some  cases 
when  events  occur  at  nearly  the  same  time,  the  movements  in 
time  may  be  very  short;  at  other  times,  e.g.,  between  12:00 
and  4:00  a.m. ,  events  may  be  rare  and  the  time  steps  may  be 
in  hours. 

In  some  cases  it  is  not  possible  to  determine  the  next 
time  at  which  an  event  will  occur.  For  example,  if  the 
event  is  an  equipment  failure  and  repair  cannot  be  started 
due  to  the  unavailability  of  personnel  (they  are  occupied 
elsewhere) ,  it  is  not  feasible  to  attempt  to  determine  when 
the  personnel  will  become  available.  Therefore,  the  event 
will  be  classified  as  delayed  and  the  proqram  will  attempt 
to  process  it  at  each  successive  imminent  event.  That  is, 
each  time  the  clock  is  moved,  the  program  will  check  for  de¬ 
layed  events  (identified  by  a- special  flag)  and  if  any  exist, 
it  will  attempt  to  initiate  them. 

Events  which  occur  in  the  model  are  processed  by  sub¬ 
routines  which  represent  system  functions.  The  system 
functions  which  represent:  functions  performed  onboard  ships 
will  be  discussed  in  Section  III-B-1.  Events  are  processed 
in  the  model  by  the  use  of  transactions  which  are  contained 
in  a  transaction  table.  This  table  contains  the  data  about 
each  event,  such  as  its  type,  the  time  of  its  occurrence,  the 
function  or  subroutine  it  is  currently  "in"  and  the  next 
function  which  will  occur  next.  When  an  event  becomes  inf- 
minent,  the  master  program  uses  the  data  in  the  transaction 
table  to  determine  what  subroutines  should  be  used  to  pro¬ 
cess  it. 

Resources,  such  as  personnel,  which  are  required  to 
perform  a  function  are  made  unavailable  to  other  events  when 
the  event  transaction  enters  a  function.  These  resources 
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are  not  released  until  the  function  has  been  completed  (e.g., 
the  simulation  clock  has  moved  to  the  time  at  which  the 
transaction  is  scheduled  to  leave  the  function) .  In  some 
cases,  higher  priority  demands  may  interrupt  an  activity,  in 
which  case  the  resources  are  released  immediately. 
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II.  REVIEW  OF  SHIP  II  STUDIES 


A.  DDG-2  Reduced  Manning  Study  (Schwartz,  Parker  and 
Rhodes ,  1970 ) . 


1.  Study  Description 

This  study  was  done  to  evaluate  the  impact  of  a 
general  across-the-board  reduction  in  the  crew  of  a  guided 
missile  destroyer  (DDG)  on  ship  operations,  personnel  work¬ 
load  and  ship  readiness.  The  normal  ships  allowance  of  319 
men  was  taken  as  the  baseline  against  which  to  compare  the 
reduced  manning.  The  reduced  complement  totaled  267  men 
and  was  developed  from  an  enlisted  distribution  plan  pro¬ 
vided  by  OPNAV.  The  267  men  represents  a  reduction  of 
approximately  16%.  Two  simulation  runs  of  10  weeks  each 
were  used  for  each  of  the  two  conditions  (baseline  manning 
and  reduced  manning) .  This  is  equivalent  to  two  replica¬ 
tions,  each  of  sample  size  10,  for  each  condition. 

The  following  changes  were  made  in  the  billets  allotted 
to  each  division  of  the  DDG-2: 

N  Division  • 


Deletions : 

Ships  Personnel  Clerk  (1) 
OC  Division 


Deletions : 

Electronic  Communication  Tech  (1) 
Radar  Tech  (1) 

Radarman  (10) 

Additions : 

Radarman  Apprentice  (2) 

OE  Division 


Additions : 

Electronics  Tech  (1) 
2nd  Division 


Deletions : 
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Missile  Ordinance  Tech  (1) 
Gun  Ordinance  Tech  (2) 

Additions : 

Gun  Ordinance  Trainee  (1) 

F  Division 


Deletions : 

Fire  Control  Tech  (1) 

Apprentice  Fire  Control  Tech  (6) 

F-2  Division 

No  Changes 

1st  Division 

Deletions : 

Boatswains  Mate  (2) 

Apprentice  Boatwains  Mate  (3) 
Facilities  Mainteranceman  (1) 

M  Division 

Deletions : 

Assistant  Propulsion  Supervisor  (2) 
Propulsion  Machine  Operator  (3) 
Propulsion  Machine  T/E  (1) 

B  Division 


Deletions : 

Ass't  Steam  System  Supervisor  (1) 
Steam  System  Operator  (1) 

Apprentice  Steam  System  Operator  (1) 
Steam  System  Trainee  (1) 

R  Division 


No  change 
S  Division 


Deletions : 
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Supply  Accountant  (1) 
Storekeeper  (1) 

Storekeeper  Apprentice  (1) 

Pay  Records  Adm.  Apprentice  (1) 
Ships  Storeman  (1) 

Ships  Launjlryman  (1) 

Ships  Cook  (1) 

Apprentice  Cook  (2) 

Food  Service  Trainee 
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2 .  Findings 

Reducing  the  crew  size  by  16%  led  to 
slight  increases  in  personnel  workload  and  a  marked  inabi¬ 
lity  to  accomplish  the  scheduled  daily  work  requirements. 

Most  of  the  high  priority  tasks  (i.e.,  corrective  maintenance, 
evolution)  were  accomplished  but  some  personnel  were  highly 
overworked.  Delays  in  repair  and  troubleshooting  were  fre¬ 
quent  and  there  was  a  moderate  decrease  in  division  training. 
Large  amounts  of  facilities  maintenance,  and  support  and 
administrative  work,  both  low  priority  tasks,  were  not  done 
under  the  reduced  manning.  The  results  showed  that  a  ship 
operating  under  the  reduced  manning  could  operate  properly 
for  a  short  time  but  ship  readiness  would  probably  decrease 
over  a  long  period  of  time.  This  study  showed  that  Ship  II 
is  useful  for  study  of  manning  reduction  problems. 

A  detailed  description  of  the  findings  is  contained 
in  Schwartz,  _e£.  al.  (19  70)  ,  Appendix  B.  The  findings  of  the 
study  are  summarized  in  Tables  4  and  12. 

B.  DDG-2  Sensitivity  Analysis  (Schwartz,  Parker  and  Rhodes, 
1970] 


1.  Study  Description 

This  study  was  done  to  determine  how  selected  output 
variables  of  the  model  responded  to  changes  in  independent 
variables  associated  with  equipment  maintenance  and  ships 
work.  This  study  was  one  of  the  first  attempts  to  do  sensi¬ 
tivity  testing  of  the  model.  * 


sj 
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The  purpose  of  this  study  was  to  compare  responses  of 
15  output  variables  (including  planned  and  corrective  mainten¬ 
ance,  manhours,  queues  and  length  of  time  in  queues,  readiness, 
cancelled  work,  etc.)  resulting  from  changes  in  equipment  re¬ 
liability  (mean  time  between  failures)  and  repair  and  trouble¬ 
shooting  task  times  (Time-in-Function) .  The  mean  time  between 
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failure  (MTBF)  and  time  in  function  (TIF)  values  were  varied 
using  Parametric  Variation  Ratios  (See  Section  III-F) .  Four 
values  of  each  input  variable  were  used,  a  baseline  value 
plus  three  variations  from  baseline.  For  MTBF,  the  varia¬ 
tions  from  baseline  were  20%,  40%  and  60%  increase  in  MTBF 
for  all  equipment  items  of  the  ship.  TIF  variations  were 
20%,  40%  and  60%  decreases  in  TIF  values.  For  each  the  four 
conditions  of  each  variable,  two  replications  of  a  10  weeks 
each  were  run. 

2 .  Findings 

The  effect  of  variation  in  TBF  and  the  TIF  was  small 
as  measured  by  the  chances  in  the  15  model  output  variables 
studied.  Variability  of  tempo  of  operation  appeared  to  exceed 
the  influence  of  changes  ir.  TBF  and  TIF.  The  result  of 
varying  TIF  is  possibly  more  evident  when  presented  graphically, 
but  none  of  the  differences  between  BL,  TBF  and  TIF  conditions 
were  statistically  significant  at  the  0.1  level.  The  lack  of 
practical  and/or  statistical  differences  in  this  analysis 
could  result  from  a  model  which  is  insensitive,  a  poor  choice 
of  parameters  to  vary,  a  choice  of  unsuitable  output  data 
categories  to  study,  or  too  small  of  a  sample  size. 

Schwartz,  et.al.  (1970),  Appendix  C,  contains  a  de¬ 
tailed  description  of  the  study  outputs.  Tables  6,  7,  14 
and  15  summarize  the  findings  from  this  study. 

C.  Ship  II  Scenario  Study  (Schwartz,  1971) 

1.  Study  Description 

All  previous  applications  of  Ship’  II  had  been  total 
ship  simulations;  i.e.  they  used  all  ship  divisions.  The  in¬ 
dependent  variables  in  these  studies  were  varied  within  only  a 
small  part  of  their  total  range.  This  study  was  based  on 
recognition  of  a  need  for  more  quantitative  and  sophisticated 
evaluation  of  the  response  characteristics  of  ship  II.  The 
study  was  also  intended  to  demonstrate  the  potential  utility 
of  Ship  II  to  Navy  manpower  planners. 

• 

Specifically  the  objectives  of  the  study  were: 

a.  To  test  the  applicability  of  ship  II  to  a  ship 
division  rather  than  a  total  ship,  using  a  smaller  data  set 
with  greater  detail.  The  ASW  Division  of  a  DD  963  destroyer 
was  used. 
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b.  To  study  the  sensitivity  of  a  number  of  output 
measures  to  systematic  changes  in  the  input  variable  "Scenario 
Load."  Scenario  load  is  defined  as  the  time  spent  in  evolu¬ 
tions,  training  exercises,  and  Condition  I. 

Four  levels  of  Scenario  load  were  used: 

1.  No  load  -  Readiness  Condition  IV. 


For  this  condition  no  external  operational  re¬ 
quirements  were  imposed  on  the  ship.  Only  Condition  IV  watch 
stations  were  manned.  All  internal  requirements  (ships  work, 
division  training,  etc.)  were  scheduled. 

2.  Low  load  -  Readiness  Condition  IV. 


A  low  load  scenario  was  designed  in  cooperation 
with  the  DD  963  Project  Office.  The  average  scenario  load 
hours  programmed  was  12.96  hours/week,  or  44%  of  the  wartime 
load  (Scenario  4)  . 

3.  Normal  Peacetime  -  Readiness  Condition  IV. 

The  normal  peacetime  scenario,  developed  with  the 
aid  of  the  Project  Office,  included  21.63  hours/week  of 
scenario  events  (73%  of  the  wartime  scenario  load). 

4.  Wartime  load  -  Readiness  Condition  III. 


For  this  condition  a  combat  scenario,  including 
enemy  engagements,  was  developed.  The  average  scenario  load 
was  29.50  hours /week. 

Each  scenario  was  a  run  for  five  replications  of  each 
of  size  10  weeks. 

2 .  Findings 


Of  the  34  dependent  variables  used  (representing 
workload,  watch  hours,  division  training,  evolution,  PM,  CM, 

FM  and  SA)  all  but  five  were  found  to  be  sensitive  to  varia¬ 
tion  in  scenario  load.  In  addition,  the  direction  of  change 
in  most  of  the  dependent  variables  was  consistent  with  the 
expectations  of  the  experimenters.  For  a  detailed  presenta¬ 
tion  of  output  variables,  consult  Schwartz  (1971).  A  summary 
of  the  study  findings  is  contained  in  Tables  5  and  13. 
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D.  NEC  Reduction  Study  (Schwartz,  1971) 


1.  Study  Description 

In  this  study,  the  independent  variable  was  number 
of  persons  in  the  ASW  pivision  qualified  (via  Naval  Enlisted 
Classifications  (NEC))  to  perform  a  set  of  maintenance  tasks 
associated  with  the  AN/SQS-26  CX  Sonar  System.  The  total 
number  of  ASW  Division  personnel  remained  the  same  but  their 
qualifications  were  changed,  i.e.,  four  0483  NEC's  were 
deleted.  The  scenario  was  the  Wartime  Scenario,  used  in  the 
previous  study  (Study  C). 

Five  dependent  variables  were  studied:  Total  workload 
for  watchstanders ,  non-watchstanders  and  total  division,  number 
of  PM  jobs  cancelled  per  week  and  ships  work  manhours  re¬ 
maining  undone  per  week.  Each  of  the  two  levels  of  the  inde¬ 
pendent  variable  was  run  for  5  replications  of  10  weeks  each. 

2.  Findinqs 


The  reduction  in  number  of  men  with  NEC  of  0483  had  no 
statistical  or  practical  significance  as  measured  by  values 
of  the  five  dependent  variables.  Given  that  the  reliability 
estimates  of  equipments  (AN/SQS-26CX)  are  accurate,  it  is 
reasonable  to  assume  no  degradation  in  capability  or  ability 
to  maintain  equipment  would  result  from  having  five  vice  nine 
maintenance  men  with  NEC  0483  aboard  the  DD  963.  This  study 
further  demonstrated  the  utility  of  Ship  II, 

For  more  details  of  this  study,  consult  Schwartz 
1971,  Appendix  E. 

E.  Personnel-Workload  Two  Factor  Study  (Schwartz  and  Harris, 
1972) 


1.  Study  Description 

Previous  applications  of  Ship  II  had  focused  on  the 
relationships  between  output  (dependent)  variables  and  single 
input  (independent)  variables,  let  one  of  the  major  reasbns 
for  using  simulation  modeling  is  to  study  the  effects  of 
interactions  between  independent  variables,  which  are  not 
easily  represented  analytically.  This  study  was  the  first 
attempt  to  consider  the  effects  of  simultaneous  variations 
of  Ship  II  independent  variables. 

The  specific  problem  addressed  in  the  study  was  to 
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explore  the  relative  effects  of  variations  in  manpower  level 
and  imposed  workload  demand  upon  workload  and  task  accomplish¬ 
ment.  Manpower  level  and  workload  demand  thus  comprised  the 
two  independent  variables  used  in  the  study.  Five  levels 
were  defined  for  each  variable,  resulting  in  25  combinations 
of  values.  A  single  replication  of  5  weeks  in  length  was  run 
for  each  of  the  25  values  of  the  independent  variables. 

Manpower  level  was  defined  as  the  total  number  of  2nd 
and  3rd  class  Sonar  Technicians  (STG)  in  the  ASW  Division  of 
the  DD963.  The  five  levels  for  this  variable  were  6, 5, 4, 3, 2, 
Sonar  Technicians.  Workload  demand  was  defined  as  the  total 
FM  and  S/A  workload  imposed  on  the  STGs.  The  levels  for  this 
factor  were  100%,  80%,  60%,  40%,  and  20%  of  the  FM  and  S/A 
workload  specified  in  the  DD963  Ship  Manning  Document. 


2.  Findings 


Six  of  the  15  dependent  variables  were  sensitive  to 
variation  in  Manpower  level  and  9  were  not  sensitive.  For 
workload  level,  this  variable  was  not  expected  to  have  any 
effect  on  12  of  the  independent-dependent  variable  relation¬ 
ships.  The  other  three  dependent  variables  were  found  to  be 
sensitive . 


Judgements  were  made  for  each  dependent-independent 
variable  combination  as  to  whether  the  results  obtained 
were  consistent  with  the  expectations  of  the  experimenters. 
Twenty-eight  such  combinations  wer  defined.  For  13  of  these 
combinations,  the  data  were  judged  to  be  inconclusive.  Of 
the  remaining  15  combinations,  12  were  judged  to  lend  .support 
for  model  validity  and  three  were  judged  not  to  support  model 
validity.  A  detailed  presentation  of  study  outputs  is  con¬ 
tained  in  Schwartz  and  Harris,  (1972)  .  The  results 
of  this  study  are  summarized  in  Tables  8,  9,  16  and  17. 
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DD-963  Reduced  Manning  Study  (Rhodes,  1972) 


Study  Description 


The  possibility  of  an  all  volunteer  Armed  Forces  hac 
caused  Navy  manpower  planners  to  consider  the  prospect  of  a 
significantly  reduced  personnel  pool.  Analysts  must  assess 
the  impact  of  personnel  reduction  and  rlan  ways  to  meet  the 
service  and  operational  requirements  with  the  reduced  number 
of  men.  One  possible  way  to  the  reduce  shipboard  manning  re¬ 
quirement  is  to  move  some  shipboard  functions  ashore,  where 
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they  might  be  performed  more  efficiently.  Ship  II  has  been 
proposed  as  a  way  to  evaluate  this  possibility.  A  set  of 
simulation  runs  could  provide  data  to  asr ess  the  effects  of 
reduced  task  requirements  on  manDower  ut:  lization  and  the 
effects  of  reduced  manning  on  task  perfoimance. 

A  three  part  iterative  method  was  used  in  the  study. 

1.  A  baseline  run  established  the  statistics  on 
manning  and  workload  for  the  ship  as  currently  manned. 

2.  A  reduced  task  run  produced  the  statistics  des¬ 
cribing  workload  and  task  performance  based  on  full  manning 
but  with  a  selected  set  of  tasks  removed.  Presumably,  indi¬ 
vidual  workload  level  for  certain  crewmen  would  decrease 
since  the  work  requirements  decreased- 

3.  A  reduced  manning- reduced  task  run  produced  sta¬ 
tistics  describing  workload  and  task  performance  with  both 
manpower  and  task  requirements  reduced.  The  statistics  were 
then  evaluated  to  determine  if  the  reduced  crew  can  accomplish 
the  renu.ining  tasks  adequately. 

The  studv  was  done  with  the  S  Division  of  a  DD-963. 
After  the  baseline  run  it  was  decided  that  64  ships  work 
tasks,  totalling  316  hours  per  week  could  be  performed  more 
efficiently  ashore.  The  deleted  tasks  were  primarily  o*  the 
following  types:  administrative  paper  work,  bakinq,  shi  .s 
store,  laundry,  barber  and  disbursing. 

After  Ship  II  was  run  with  these  tasks  eliminated,  it 
was  determined  that  8  men  could  be  removed  from  the  ship. 

The  remaining  tasks  which  were  assigned  to  these  men  wore 
assigned  to  other  remaining  men.  The  simulation  was  then  run 
with  both  reduced  tasks  and  reduced  manning.  Three  criteria 
were  used  to  evaluate  the  reduced  manning:  (1)  All  watch 
stations  must  be  manned,  (2)  task  can  be  performed  by  higher 
rated  men  than  originally  assigned  but  not  by  lower  rated  men, 
(3)  Average  workload  level  under  reduced  manning  -  reduced 
task  conditions  should  approximate  the  level  under  full  manning- 
full  tasks. 


Findings 


All  three  criteria  were  satisfied  by  the  reduced  task- 
reduced  manning  run.  The  study  showed  the  S  Division  could 
operate  with  8  fewer  men  if  64  of  the  14S  ships  work  tas>n  are 
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moved  ashore.  The  study  also  demonstrated  that  ship  II  is 
a  good  tool  to  use  in  gathering  data  necessary  to  evaluate 
manning  reduction  proposals.  Rhodes  (1972)  contains  de¬ 
tailed  presentation  of  the  data.  Tables  10  and  18 
contain  summaries  of  study  findings. 


III.  EVALUATION  OF  SHIP  II 


A.  Evaluation  Criteria 


One  of  the  problems  in  evaluating  a  computer  simulation 
models  to  select  the  set  of  criteria  to  be  used.  The  cri¬ 
teria  used  in  this  paper  are  based  on  three  studies  which  have 
attempted  to  specify  an  integrated  model  evaluation  criteria 
set  (Meister,  1971;  Rhodes,  1970;  and  Hutchins,  Prather, 
Barefoot  &  Flint,  1973) .  The  criteria  will  be  listed  and  de¬ 
fined  here;  a  brief  discussion  of  each  criteria  is  contained 
in  Sections  B  through  G,  along  with  the  findings  relating  to 
each  criteria. 

1.  Validity 

Two  types  of  validity  will  be  considered:  construct 
validity  and  face  validity.  Face  validity  is  a  form  of  pre¬ 
dictive  validity  which  will  be  used  because  of  the  lack  of 
data  on  Ship  II  predictive  validity.  The  purpose  of  both 
types  of  validity  is  to  indicate  the  correspondence  of  model 
outputs  with  real  world  data.  Construct  validity  involves  a 
comparison  between  real  world  process  and  the  fidelity  with 
which  the  processes  are  represented  in  the  model. 

2.  Reliabilitv 


Two  aspects  of  this  criterion  will  be  discussed.  The 
first  is  the  ability  of  the  model  to  produce  consistent  out¬ 
puts  when  applied  to  similar  problems  by  different  users.  The 
other  aspect  of  reliability  relates  to  precision  of  model 
results. 


3.  Sensitivity 

Sensitivity  refers  to  the  extent  to  which  variation 
in  model  input  variables  causes  corresponding  variation  in 
model  output  variables. 

4.  Utility 

The  utility  of  a  model  depends  directly  on  the  useful¬ 
ness  of  the  data  produced  by  the  model  in  aiding  in  solution 
of  real  world  manpower  and  personnel  problems. 


5.  Practicalit 


This  criterion  refers  to  factors  which  constrain  or 
facilitate  the  use  of  a  model,  such  as  cost  of  using  the 
model,  input  data  preparation  reouired,  and  other  such  factors. 

6.  Other  Criteria 


Included  in  this  category  are  the  criteria  of  System 
Development  Applicability  and  Model  Structure.  In  addition, 
the  responses  given  for  Ship  II  to  a  model  evaluation  ques¬ 
tionnaire  will  be  presented. 

B.  Validity 

1.  Discussion  of  Concept 


Validity  is  perhaps  the  most  important  criterion  by 
which  to  evaluate  a  simulation  model.  Since  computer  simula¬ 
tions  are  intended  to  represent  and  predict  real  life  situa¬ 
tions,  the  degree  to  which  model  outputs  agree  with  corre¬ 
sponding  variables  in  the  real  world  should  be  a  major  criteria 
for  model  evaluation. 

The  literature  indicates  that  two  types  of  validity 
exist:  predictive  and  construct.  Predictive  validity  refers 
to  the  degree  of  agreement  between  the  output  values  produced 
by  the  model  and  the  values  of  the  same  variables  in  the  real 
world  under  the  same  or  equivalent  conditions.  Construct 
validity  will  be  defined  as  the  degree  of  agreement  between 
the  processes  of  the  model  and  the  corresponding  real  world 
processes  being  modeled. 

a.  Predictive  Validity.  Proof  that  a  model  has  made 
correct  predictions  is  probably  the  most  convincing  evidence 
that  a  model  is  valid.  However,  a  major  logical  problem  exists 
when  validating  a  model  by  this  means.  When  a  model  has  been 
exercised  with  a  particular  combination  of  values  of  input 
variables  and  found  to  be  valid,  the  finding  holds  only  for 
those  and  similar  input  variables.  The  fact  that  a  model  is  valid 
for  a  particular  combination  of  inputs  does  not  guarantee  that 
it  will  be  valid  for  other  combinations.  In  addition,  even 
for  a  single  input,  that  fact  that  a  model  output  is  valid 
for  a  particular  value  of  an  input  variable  does  not  guarantee 
that  the  model  is  valid  over  the  entire  range  of  values  of  that 
variable. 
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The  situation  can  become  even  more  complex  when 
input-output  relationships  are  considered.  A  complex  model, 
such  as  Ship  II,  usually  has  many  different  input  and  output 
variables.  Ship  II,  for  example,  has  approximately  30  major 
input  variables  and  30  major  output  variables.  The  validation 
process  then  becomes  the  task  of  testing  combinations  of  input 
and  output  variables  against  an  external  criteria.  The  impos¬ 
sibility  of  testing  such  combinations  exhaustively  with  a  model 
the  size  of  Ship  II  should  be  immediately  clear.  Siegel  (1973)  , 
for  example,  points  out  that  "a  model  such  as  Ship  II  provides 
such  a  rich  variety  of  output  data  that  validation  of  the  pre¬ 
dictions  of  each  output  category  represents  a  rather  formidable 
and  impractical  task.  Nevertheless,  the  validity  concept 

must  be  included  in  any  discussion  of  a  model  which  is  under 
consideration  for  use  as  a  tool  for  providing  the  decision 
maker  with  information  which  will  help  him  to  reach  the  re¬ 
quired  decisions."  (p.  5) 


b.  Face  Validity.  No  predictive  validity  studies  have 
been  performed  with  Ship  II.  But  face  validity  information  is 
available.  Face  validity  is  defined  by  Hermann  (1967)  as  "a 
surface  or  initial  impression  of  a  simulation's  realism."  (p.  221.] 
Face  validity  is  a  degraded  form  of  predictive  validity  and  is 
based  on  human  judgements  of  the  degree  to  which  model  outputs 
correspond  with  .’reality,  instead  of  statistical  comparisons 
of  model  outputs  with  a  criterion..  Face  validity  is  certainly 
no  substitute  for  predictive  validity,  but  in  the  absence  of 
predictive  validity,  it  does  provide  some  information  about  , 

model  quality.  Instead  of  predictive  validity,  Ship  II  will 
be  evaluated  against  the  criteria  of  face  validity. 


c.  Construct  Validity.  As  defined  above,  construct 
validity  refers  to  the  extent  of  agreement  between  processes 
of  the  model  and  the  corresponding  real  world  processes  being 
modeled.  The  process  by  which  construct  validity  is  deter¬ 
mined  is  mainly  human  judgement  based  on  comparisons  between 
the  modeled  functions  and  the  functions  as  actually  performed. 

Evidence  foi  predictive  validity  provides  strong 
support  for  a  model  but,  as  stated  above,  has  very  limited  | 

generality  to  other  inputs  or  outputs  not  tested.  Construct 
validity,  on  the  other  hand,  has  comparatively  wide  generality. 
Construct  validity  provides  evidence  that  a  model  will  operate 
correctly  over  a  wide  range  of  innuts,  outputs  and  combinations 
of  inputs  and  outputs.  A  well  done  construct  validation  gives 
more  assurance  that  a  model  will  operate  correctly  than  does 
a  predictive  validation  study. 
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Construct  validity  studies  are  important  in  model 
validation  also  because  of  the  difficulty  in  performing  pre¬ 
dictive  validity  studies.  Hatch  (1967)  ,  for  example,  feels 
that  in  our  experience  statistical  validation  of  simu¬ 

lation  models  is  often  impractical.  This  is  true  because: 

(1)  real  world  data  tyoically  is  unavailable,  (2)  the  cost 
of  collecting  the  necessary  data  and  running  the  model  is  very 
high,  (3)  the  various  statistical  techniques  are  often  of 
questionable  value,  and  (4)  such  exercises  do  not  guarantee 
validity  for  untested  situations.  Therefore,  major  emphasis 
should  be  placed  on  validation  of  the  model  concept."  (p.  177) 

2.  Ship  II  Validity 

a.  Construct  Validity. 


(1)  Description  of  Ship  II.  As  stated  in  the  pre¬ 
vious  section,  evaluation  of  construct  validity  is  based  mainly 
on  human  judgement.  Such  an  evaluation  has  been  performed  for 
Ship  II  and  is  reported  in  Section  (3) .  However,  the  judge¬ 
ments  on  which  construct  validity  was  based  are  subjective  and 
the  rationale  for  the  judgements  was  not  explicitly  documented. 
Therefore,  the  next  section,  a  detailed  description  of  ship¬ 
board  functions  modeled  in  Ship  II,  is  intended  to  provide  the 
reader  with  enough  information  to  make  his  own  rough  evaluation 
of  Ship  II  construct  validity.  The  Ship  II  flow  diagram  in 
Figure  1  shews  that  the  basic  model  functions,  their  input  and 
output  states,  and  their  interrelationships.  Each  block  dia¬ 
gram  in  Figure  1  represents  a  shipboard  function  which  is  in¬ 
cluded  in  Ship  II.  Section  (2)  contains  a  description  of  each 
of  the  functions  shown  in  Figure  1. 

•  1 

(2)  Description  of  Ship  II  Functions. 


(a)  Analyze  ‘'.Derations  Requirements  (ANOPREQ)  . 
The  purpose  of  this  function  is  to  process  the  commands  entered 
in  the  employment  schedule  (scenario).  Each  command  has  an 
execution  time  specified  by  the  user  and  when  this  time  arrives 
this  function  initiates  the  action  necessary  to  respond  to  the 
command,  including  watch  changes,  ship  status  change  (sea/port) 
and  changes  in  the  overall  readiness  condition. 


types  of  commands: 


The  scenario  can  specify  the  following 


Some  of  the  material  in  this  section  is  based  on  Smith, 
Spencer  &  Brooks  (1970) . 
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•  Sea/port  changes  and  durations; 

•  Evolutions; 

•  Training  exercises; 

•  Imposed  readiness  (actual  Condition  III 
or  I)  ; 

•  Output  reports 

(b)  Assign  Watch  (ASNWATCH) .  Watch  and  evo¬ 
lution  station  manning  is  based  on  the  general  readiness  status 
of  the  ship  and  the  requirement  to  perform  exercises  or  evo¬ 
lutions.  The  basic  readiness  condition  of  the  ship,  as  speci¬ 
fied  by  the  employment  schedule,  can  be:  (a)  Condition  I 
(General  Quarters)  ,  (b)  Condition  III  (Wartime  Steaming) ,  (c) 
Condition  IV  (Peacetime  Steaming)  or  (d)  Condition  V  (in-port) . 
These  basic  conditions  are  set  by  assigning  personnel  to  watch 
stations  in  accordance  with  input  data.  Watch  stations  are 
manned  twenty-four  hours  a  day  by  rotating  the  personnel. 

With  the  exception  of  Condition  I,  each 
watch  station  has  three  sections  and  personnel  work  on  a  two 
or  four  hours  on,  and  six  or  eight  hours  off  cycle.  The  Navy 
follows  the  practice  of  "dogging"  watches,  which  means  that 
sections  of  the  watch  shift  so  that  any  individual  will  not 
stand  the  same  watch  every  day. 

Watch  assignments  are  made  using  the 
current  readiness  condition  and  section  based  on  time  of  day. 
This  information  allows  the  identification  of  the  specific 
individuals  who  are  to  stand  watch.  Each  man  required  for 
watchstanding  is  placed  in  the  "on-watch"  category  at  the 
appropriate  time  and  those  who  were  on  watch  are  place-  in 
the  "idle"  category  at  the  same  time.  If  a  man  is  performing 
a  job  (other  than  an  evolution)2  when  his  watch  time  arrives, 
that  job  is  interrupted  so  that  he  may  be  released  to  perform 
the  evolution. 


Training  exercise  requirements  included 
in  the  employment  schedule  may  specify  augmentation  to  the 
basic  watch  condition  and  these  are  also  handled  by  this 
function.  For  example,  if  an  exercise  only  involves  Anti- 

- - - - 

evolutions  are  the  highest  priority  activity  in  the  model 
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Submarine  Warfare  (ASW)  stations,  the  basic  watch  may  be  aug¬ 
mented  such  that  only  the  ASW  stations  are  manned  at  Condition 
I.  Input  data  allows  the  user  to  specify,  through  personnel 
assignments,  up  to  five  watch  augmentations  for  training 
exercises.  In  completing  the  employment  schedule,  the  user 
specifies  which  of  these  exercises  will  occur,  and  specifies 
which  stations  are  augmented  for  specific  exercises. 


(c)  Assign  Planned  Maintenance  (AJSPMS). 

This  function  reviews  requirements  for  planned  maintenance  and 
assigns  required  tasks  to  available  personnel.  The  mainten¬ 
ance  task  load  is  based  on  the  Planned  Maintenance  System 
(PMS)  and  covers  all  tasks  required  aboard  ship  with  the  ex¬ 
ception  of  those  performed  during  overhaul  and  those  performed 
less  frequently  than  once  a  year. 


THE  ASSIGN  PLANNED  MAINTENANCE  function 
is  executed  daily  during  the  course  of  a  simulation  run.  The 
program  reviews  the  PM  table  tr  identify  PM  actions  that  have 
either  been  delayed  or  are  scheduled  to  arise  within  six  hours 
after  the  start  of  the  day.  The  workday  within  which  PM  may 
be  scheduled  is  specific  by  input  data.  Daily,  weekly,  monthly, 
quarterly,  semi-annual  and  annual  PM  jobs  are  represented 
individually. 


When  a  PM  job  enters  the  function,  a  test 
is  made  to  determine  if  there  are  any  constraining  conditions 
which  prevent  performance  of  the  task.  These  constraining 
conditions  are: 


(1_)  Certain  PM  tasks  specified  by  the 
user  are  performed  only  in-port; 


(2)  No  PM  tasks  are  performed  during 


Condition  I; 


(^)  PM  is  assigned  only  during  specified 
working  hours  and  not  on  Sunday  (except  daily  jobs);  and. 


i 

i 


-  ( 4 _)  If  the  equipment  on  which  PM  is  per¬ 
formed  is  "down",  the  job  is  delayed.  • 

If  a  PM  action  cannot  be  performed  due  to 
one  of  the  above  constraints,  it  is  delayed  or  postponed  until 
the  following  day.  The  delayed  action  is  listed  in  a  special 
table  for  subsequent  processing.  If,  however,  the  PM  task  can 
be  performed,  this  function  obtains  the  list  of  required 
P^-*--onnel  (which  is  specified  by  the  user)  and  scans  the  personnel 
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status  table  to  locate  the  required  personnel.  A  man  cannot 
be  assigned  to  planned  maintenance  unless  he  is  either  idle 
or  performing  ship's  work.  In  addition,  multi-man  PM  jobs 
cannot  be  started  until  all  specified  personnel  are  available. 
When  personnel  requirements  have  been  met,  the  selected  men 
will  be  moved  to  the  PM  category  in  the  personnel  status  table. 
Once  persnnel  have  been  assigned,  the  job  is  sent  to  PERFORM 
PLANNED  MAINTENANCE.  If  the  required  personnel  cannot  be  ob¬ 
tained,  the  job  is  delayed  until  they  are  available,  or  upward 
absorption  is  used  if  specified  by  input  data. 

(d)  Perform  Planned  Maintenance  (PERPMS) .  This 
function  represents  the  execution  of  Planned  Maintenance  Tasks. 
Function  execution  is  characterized  hv  the  time  reouired  for 
completion  of  the  PM  task  and  the  probability  that  existing 
eauipment  failures  are  detected.  The  total  performance  time 
includes  make-ready  and  put-away  time  in  addition  to  basic 
task  time. 

When  a  PM  job  enters  this  function,  the  system 
being  maintained  is  checked  to  determine  if  an  undetected  fail¬ 
ure  exists.  If  so,  whether  or  not  the  failure  is  detected  is 
determined  using  a  probability  value  associated  with  the  system 
and  the  type  of  PM  action.  If  detection  occurs,  the  model 
transfers  to  the  troubleshooting  function.  If  not,  the  PM 
function  continues. 

Since  PM  jobs  can  be  interrupted  and  per¬ 
sonnel  reassigned  to  higher  priority  activities,  such  as  crit¬ 
ical  corrective  maintenance,  watch  assignments,  or  General 
Quarters,  the  PM  function  can  reprocess  jobs  which  are  re¬ 
entered  following  completion  of  the  interrupting  activity. 

At  the  time  the  PM  job  is  interrupted  the  system  will  store 
the  remaining  time  to  complete  the  PM  task.  The  interrupted 
PM  transaction  will  later  be  processed  as  a  normal  PM  trans¬ 
action  and  the  time  remaining  which  was  stored  when  the  job 
was  interrupted  will  be  used  as  the  task  time. 

Since  the  Planned  Maintenance  System  is  de¬ 
signed  to  prevent  as  well  as  detect  malfunctions,  any  delay 
or  cancellation  of  P.MS  jobs  may  adversely  affect  the  reliability 
of  hardware.  For  future  model  growth  capability,  each  PM  task 
can  be  assigned  a  code  of  P,  C,  or  B  which  denotes,  respec¬ 
tively,  whether  the  action  is  a  task  to  determine  the  state 
of  the  equipment  (P) ,  an  action  which  changes  the  state  of 
the  equipment  (C)  or  both  (b) .  If  actions  coded  C  or  B  are 
delayed  for  a  significant  period  of  time,  it  is  likely  that 
the  equipment  will  degrade,  resulting  in  an  increase  in  the 
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probability  of  failure.  Presently,  the  model  has  no  provision 
for  simulating  this  effect,  and  it  is  unlikely  viable  data  on 
this  effect  will  be  available  for  some  time  to  come. 

(e)  Assign  Daily  Facilities  Maintenance  and 
Administrative  and  Support'  Work.  This  function  checks  the 
time  of  day,  the  day  of  the  week ,  the  port  or  sea  state,  and 
the  readiness  condition  to  determine  if  the  list  of  Facilities 
Maintenance  (FM)  and  Support/Administrative  (S/A)  work  should 
be  scheduled.  If  all  conditions  are  met,  then  the  jobs  are 
scheduled.  If  any  condition  is  not  met,  scheduling  is  post¬ 
poned  to  a  later  time.  This  may  involve  work  being  carried 
over  to  the  next  workday.  FM  and  S/A  jobs  are  scheduled 
Monday  through  Saturday  (inclusive)  while  at  sea,  and  Monday 
through  Friday  while  in  port. 

(f )  Perform  Daily  Facilities  Maintenance  and 
Administrative  and  Support  Work  (PERWORK) .  When  ships  work 
can  be  assigned,  this  function  makes  the  detailed  assignments 
to  individuals.  Before  the  job  is  assigned,  this  function 
determines  how  many  men  are  required  by  dividing  the  required 
man  hours  by  the  time  remaining  in  the  day.  Using  this  value, 
an  attempt  is  made  to  assemble  a  team  from  the  available  per¬ 
sonnel.  When  jobs  will  require  only  one  man,  this  process  is 
not  required. 


Once  a  job  is  assigned,  the  associated  per¬ 
sonnel  are  identified  as  performing  facilities  maintenance  or 
administrative/support  tasks,  and  are  held  in  this  category 
until  the  job  is  completed  or  interrupted.  Input  data  allow 
the  user  to  specify  the  use  of  upward  absorption  when  not 
enough  primary  personnel  arh  available..  Also,  FM  and  S/A 
jobs  are  assigned  first  to  those  individuals  with  the  fewest 
total  work  hours  so  as  to  balance  the  workload . 

(g)  Schedule  Divisional  Training  (INITRA) . 

This  term  is  used  to  describe  divisional  group  meetings  which 
are  held  for  the  purpose  of  general  training.  The  types  of 
subjects  covered  in  such  meetings  include  general  orientation. 
Navy  doctrine,  leadership,  and  other  such  subjects.  This  func¬ 
tion  schedules  classes  for  each  of  the  divisions  aboard*ship, 
based  on  the  following  constraints: 

(1.)  Divisional  training  is  scheduled  only 
during  a  specified  daytime  period; 


(2)  Classes  are  not  scheduled  on  Sunday; 


(3)  Scheduling  of  classes  is  not  attempted 
during  an  actual  or  exercise  Condition  I; 

(4)  If  the  number  of  personnel  available 
for  divisional  training  is  less  than  the  minimum  specified  by 
input  data,  the  session  is  delayed  (rescheduled) . 

Ship  II  also  uses  input  data,  such  as  min¬ 
imum  number  of  personnel  required  for  a  class,  to  determine 
whether  classes  can  be  scheduled. 

If  these  above  criteria  are  met,  this  func¬ 
tion  transfers  control  to  the  EXECUTE  DIVISIONAL  TRAINING  for 
the  actual  assignment  of  personnel.  If  not,  the  model  attempts 
to  schedule  training  at  a  later  time. 

(h)  Execute  Divisional  Training  (DIVTRA) ,  When 
the  above  function  has  established  that  a  training  session  can 
be  held,  this  function  assigns  the  appropriate  personnel  to 
the  class  and  keeps  them  in  that  status  until  the  designated 
time  has  elapsed.  Since  divisional  training  can  be  inter¬ 
rupted  by  higher  priority  demands,  this  function  contains  the 
logic  necessary  to  release  personnel  and  resume  training  at  a 
later  time. 


(i)  Monitor  Watch  Station  (MOWSTA) .  This  sub¬ 
routine  computes  the  time  required  for  detection  of  failures 
in  subsystems.  When  failures  occur,  this  function  is  entered 
at  the  time  of  initial  failure  or  after  a  Type  II  error  in  the 
troubleshooting  function  (see  section  (m) ) .  The  computed  detec¬ 
tion  time  is  based  on  a  detection  delay  time  (input  by  the  user) 
which  depends  on  the  subsystem  in  which. the  error  is  present 
and  the  ship  readiness  condition.  Since  failure  detection  is 
also  possible  during  PM,  the  delay  time  computed  in  MOWSTA 

may  be  reduced,  to  represent  detection  during  PM.  This  is 
done  by  determining  if  a  PM  detection  did  occur  and  then  re¬ 
ducing  the  computed  detection  time  accordingly. 

(j)  Perform  Evolution  (PERFEVOL) .  Evolutions 
are  scheduled  when  specified  in  the  Employment  Schedule.  At 
this  time  the  PEREVOL  function  mans  the  necessary  stations 
with  the  personnel  designated  in  the  watch  bill.  This  func¬ 
tion  then  holds  these  personnel  on  station  for  the  designated 
evolution  period.  When  required  personnel  are  engaged  in 
other  tasks,  they  are  interrupted  and  the  personnel  assigned 
to  the.  evojution.  Following  the  completion  of  the  evolution, 
this  transaction  releases  the  personnel. 
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(k)  Training  Exercises  (PERTREX) .  This  func¬ 
tion  executes  the  exercises  a..d  drills  listed  in  the  Employ¬ 
ment  Schedule.  All  required  personnel  are  placed  in  training 
exercise  status  by  ANSWATCH  prior  to  entrance  into  this  func¬ 
tion.  The  time  for  completion  of  this  function  is  based  on 
input  data  in  the  Employment  Schedule. 

This  function  can  be  delayed  if  the  systems 
required  to  perform  an  exercise  (as  specified  by  input)  are 
not  available.  For  example,  if  the  user  speciries  that  a  mis¬ 
sile  fire  control  system  is  reauired  for  a  tracking  exercise, 
that  exercise  will  not  be  started  if  the  fire  control  system 
is  down  for  repairs.  Extensive  delays  for  systems  can  cause 
the  exercise  to  be  canceled. 

(l)  Repair  of  equipment  (REPAIR) .  This  func¬ 
tion  represents  the  repair  of  a  system  failure.  This  function 
is  entered  whenever  an  apparent  system  failure  has  been  found 
in  an  item  which  is  repairable  aboard  ship.  The  performance 
of  this  function  is  characterized  by  resource  requirements, 
time  to  complete  the  task  and  probability  that  the  repair  was 
performed  correctly.  Resource  requirements  are  of  two  types: 
Personnel  and  equipment.  Both  are  defined  by  the  user.  Per¬ 
sonnel  are  specified  in  terms  of  rate,  rating  and  NEC;  equip¬ 
ment  is  specified  in  terms  of  specific  equipment  items  needed. 
Performance  of  the  repair  function  will  be  delayed  whenever 
the  specified  personnel  or  equipment  are  not  available.  By 
use  of  upward  absorption  and  eauivalent  NECs,  a  secondary 

set  of  personnel  can  be  assigned  if  the  primary  set  is  un¬ 
available. 

Performance  time  is  determined  by  Ship 
II  by  making  a  random  draw  from  a  log  normal  distribution. 

The  mean  and  standard  deviation  of  the  distribution  are  speci¬ 
fied  by  the  user  for  each  equipment.  Performance  time  can, 
in  addition,  be  affected  by  the  occurrence  of  one  of  the  two 
types  of  errors  which  can  be  made  during  the  repair  function. 

A  "type  t"  error  refers  to  delay  in,  com¬ 
pletion  of  the  repair  function  beyond  a  nominal  performance 
time  specified  by  the  user.  This  type  of  error  represents 
errors  in  performance  of  repair  which  are  discovered  before 
the  repair  has  been  completed.  The  effect  of  a  "type  t" 
error  is  an  increase  in  performance  time.  The- probability 
of  occurrence  of  a  "type  t"  error  is  specified  by  the  user 
for  each  equipment. 
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The  reliability  of  repair  is  represented 
by  the  occurrence  of  a  "type  d"  error.  This  error  represents 
failure  to  correct  the  malfunction  or  damaging  another  equip¬ 
ment  component  during  repair.  The  occurrence  of  a  "type  d" 
error  is  based  on  a  random  draw  from  a  log  normal  probability 
distribution.  When  a  "type  d"  error  occurs,  the  actual  equip¬ 
ment  state  is  set  to  "bad"  while  the  apparent  state  is  set  to 
"good".  If  a  "d"  error  does  not  occur  both  actual  and  appar¬ 
ent  state  are  set  to  "good".  The  probability  of  occurrence 
of  "type  d"  errors  are  specified  by  the  user. 

The  Repair  function  can  be  interrupted  by 
higher  priority  functions.  REPAIR  contains  the  logic  required 
to  resume  performance  of  the  function  after  the  higher  priority 
function  has  been  completed.  Except  for  repair  of  equipment 
which  has  been  designated  by  the  user  as  critical,  performance 
of  the  repair  function  also  depends  on  time  of  day,  day  of 
week  and  readiness  condition  of  the  ship.  Repair  of  critical 
equipment,  on  the  other  hand,  is  done  immediately  regardless 
of  conditions.  Table  1  shows  the  factors  which  interact  to 
affect  the  performance  ol  non-critical  repair. 

One  other  factor  affects  the  performance 
of  the  repair  function:  probability  of  deferral  for  parts. 

Ship  II  determines  randomly  whether  deferrals  occur,  ^ased 
on  a  deferral  probability  specified  by  the  user.  When  a 
deferral  occurs,  the  length  of  the  deferral  is  based  on  a 
random  draw  from  a  log  normal  distribution.  The  repair  func¬ 
tion  is  reinitiated  after  this  time  has  elapsed. 


|  (m)  Troubleshoot  (TRBSHT) .  This  function  iso¬ 

lates  the  cause  of  malfunctions  to  the  level  necessary  to 
determine  the  repair  requirement;  the  function  is  initiated 
following  the  detection  of  a  malfunction.  The  performance 
of  this  function  is  characterized  by  resource  requirements 
and  time  to  complete  the  job.  Resource  needs  are  specified 
in  terms  of  rate,  rating  and  NEC  of  personnel,  and  special 
test  equipment.  Resource  requirements  for  troubleshooting 
each  ship  equipment  are  specified  by  the  user.  If  the-  re¬ 
quired  resources  are  not  available,  this  function  will  be 
delayed  until  they  are.  Like  the  repair  function,  this  ' 
function  also  has  the  capability  for  upward  absorption  and 
use  of  equivalent  NEC  whenever  the  primary  personnel  are  not 
available. 


Performance  time  is  the  time  between  the 
initial  assignment  of  resources  and  the  point  at  which  a  con¬ 
clusion  has  been  reached  as  to  the  reauired  corrective  action, 


28 


excluding  any  interruptions.  Basic  performance  time  is  deter¬ 
mined  from  a  log  normal  distribution  who  parameters  are  speci¬ 
fied  by  the  user.  Performance  time  is  increased  whenever  a 
"type  t"  error  occurs  (see  below) .  The  function  randomly  deter¬ 
mines  if  a  "type  t"  error  occurs  and,  if  so,  increases  the 
basic  performance  time  by  a  value  obtained  from  another  draw 
from  a  log  normal  distribution.  The  use  of  the  "type  t"  error 
provides  a  means  of  incorporating  the  effects  of  functional 
decision  errors  which  are  discovered  before  the  final  conclus¬ 
ion  has  been  reached. 

Performance  reliability  is  the  percent  of 
troubleshooting  attempts  which  correctly  isolate  the  source  of 
the  malfunction.  The  following  types  of  errors  can  result 
from  a  troubleshooting  action: 

TYPE  I  -  Erroneously  designating  the  state  of  a  system 
as  bad. 

TYPE  II  -  Erroneously  designating  the  state  of  a  system 
as  good. 

TYPE  III  -  Erroneously  identifying  the  cause  of  a  mal¬ 
function  (a  combination  of  a  Type  I  error  and 
a  Type  II  error) . 

TYPE  t  -  Delaying  the  conclusion  of  a  troubleshooting 
action  beyond  some  nominal  performance  time, 
input  by  the  user. 

The  probabilities  of  each  type  of  error  are  specified  by  the 
user.  When  the  job  is  completed,  these  mair  enance  error 
probabilities  are  used  to  determine  if  an  error  has  occurred. 

If  a  Type  II  error  occurs,  the  program  goes  to  function  MOWSTA. 
If  a  Type  III  error  occurs,  a  code  is  set  denoting  that  repair 
will  not  be  successful.  If  a  Type  I  error  occurs,  the  trans¬ 
action  is  sent  to  REPAIR  even  though  the  equipment  is  actually 
good.  If  no  error  occurs,  the  transaction  is  sent  to  REPAIR 
if  the  equipment  is  actually  bad;  if  the  equipment  is  actu¬ 
ally  good,  the  program  proceeds  to  the  next  scheduled  fuaction. 

The  TROUBLESHOOT  function  may  be  inter¬ 
rupted  by  higher  priority  demands,  such  as  General  Quarters, 
if  a  critical  system  is  not  being  troubleshooted.  After  an 
interrupt  has  occurred,  the  troubleshooting  function  is  re¬ 
entered  when  the  personnel  become  available  (or  the  readiness 
condition  permits) . 
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The  performance  of  non-critical  trouble¬ 
shooting  function  depends  on  time  of  day,  day  of  week  and 
readiness  condition  in  exactly  the  same  way  as  does  repair. 

Table  1  shows  how  these  factors  affect  non-critical  mainten¬ 
ance.  Troubleshooting  of  critical  equipment  is  done  immed¬ 
iately  regardless^  of  conditions. 

(n)  Support  Activity  Maintenance  (SUPAM) .  This 
function  represents  the  support  the  ship  receives  when  an  equip¬ 
ment  repair  task  is  beyond  its  capability,  due  to  lack  of 
training,  special  equipment,  or  technical  information.  This 
function  is  characterized  by  the  time  required  to  receive  the 
technical  support  required.  This  function  is  always  entered 
from  the  TROUBLESHOOT  function. 


Input  data  provide  the  probability  that 
a  repair  action  will  be  deferred  for  assistance  and  the  mean 
and  standard  deviation  of  the  time  for  such  assistance  to  be 
received.  A  log  normal  distribution  is  used  to  determine 
waiting  time. 

At  the  conclusion  of  this  function,  it  is 
assumed  that  the  repair  has  been  finished  and  the  operational 
status  of  the  failed  item  is  set  to  "good". 


(3)  Estimation  of  Construct  Validity 


The  construct  validity  of  the  Ship  II  func¬ 
tions  summarized  above  has  been  estimated  in  a  study  per¬ 
formed  by  Schwartz,  Rhodes  &  Parker  (1970).  An  index  of  con¬ 
struct  validity  was  calculated  in  the  following  manner: 

« 

(a)  Each  shipboard  function  and  associated 
subroutines  was  evaluated,  comparing  the  manner  of  its  per¬ 
formance  to  that  of  the  same  functions  either  performed  in 
the  referrent  system  or  specified  by  Navy  doctrine.  The  sub¬ 
routines  listed  in  Table  2  do  not  directly  represent  shipboard 
functions  and  therefore  were  not  described  in  the  previous 
section.  However,  since  the  construct  validity  study  did  in¬ 
clude  these  subroutines,  the  data  will  be  reported  here. 


In  the  comparison  between  modeled  func¬ 
tion  and  shipboard  performance  of  the  function,  a  scale  for 
judgement  ranging  from  0  to  1.0  was  used  scale, values  were 
assigned  to  represent  the  extent  of  agreement  between  func¬ 
tional  performance. 
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ESTIMATION  OF 

TABLE  2 

SHIP  II  CONSTRUCT  VALIDITY 

Weight  of 

Validity 

Function 

Estimate 

Function  Name 

Wf 

Vi 

1. 

ANOPREQ 

.05 

.90 

2. 

ASNPMS 

.03 

.95 

3. 

ASN WATCH 

.03 

.98 

4. 

AS  N  WORK 

.03 

.85 

5. 

DIVTRA 

.04 

.80 

6. 

INITRA 

.03 

.80 

7. 

MOWSTA 

.03 

.85 

8. 

PERFEVOL 

.04 

.85 

9. 

PERPMS 

.07 

.95 

10. 

PERTREX 

.04 

.85 

11. 

PERWORK 

.07 

.75 

12. 

REPAIR 

.09 

.97 

13. 

SUP  AM 

.04 

.85 

14. 

TBLSHT 

.07 

.97 

Subroutine  Name3 

1. 

GETCLASS 

.03 

.80 

2. 

GETEAM/CHEKTEAM 

•  .08 

.85 

3. 

GET  GUY  S /LAZ  Y  FI RS 

.06 

.90 

4. 

MANANA/CARYOVER 

.05 

.85 

5. 

SCORETYM 

.08 

.50 

6. 

WHENDONE 

.04 

.  80 

Overall  Construct  Validity 

index  -  W  X 

V.  *  =  .  847 

These  subroutines  do  not  represent  shipboard  functions  and 
were  not  described  in  the  previous  section. 
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(b)  The  scale  values  were  then  weighted  accord¬ 
ing  to  contribution  of  the  corresponding  function  to  overall 
model  validity. 

(c)  A  model  construct  validity  index  was  then 
computed  by  summing  the/  products  of  the  scaled  values  and 
function  weight,  i.e.,  CV  =  Vi  X  Wi  where 

CV  =  construct  validity  index 

Vi  =  Validity  estimate  of  the  ith  function 

Wi  =  Relative  contribution  of  the  ith  func¬ 
tion  to  overall  validity 

The  results  of  applying  the  construct 
validity  criterion  to  the  Ship  II  model  functions  are  shown 
in  Table  2.  This  table  shows  that  the  construct  validity 
index  is  satisfactory. 

Siegel  (1973) ,  after  an  independent  eval¬ 
uation  of  Ship  II,  agrees  with  this  statement:  "From  the  point 
of  view  of  the  internal  logic  of  Ship  II,  the  various  algor¬ 
ithms  appear  to  be  at  the  current  state-of-the-art"  (p.  4). 


Development. 


J4)  Construct  Validiation  Performed  During 


From  the  very  beginning  of  Ship  II  develop¬ 
ment,  there  was  great  concern  that  the  model  represent  and 
include  factors  of  interest  to  Navy  planners.  To  insure  that 
the  modeling  effort  was  properly  responsive  to  Navy  needs,  a 
Simulation  Advisory  group  was  formed  to  provide  guidance  to 
the  effort. 

One  of  the  major  decisions  made  during  the 
development  effort  was  the  selection  of  functions  to  be  in¬ 
cluded  in  the  model.  Since  the  program  was  to  simulate  an 
entire  ship  for  manpower  research,  it  was  decided  that  every 
shipboard  function  involving  human  performance  was  to  be  , 
treated.  These  functions  included  the  performance  of  watch, 
evolutions,  exercises,  division  training,  and  the  various 
classes  of  maintenance  and  ship's  work. 


See  Section  I-C. 


TABLE  3 


ASNWORK 

DIVTRA  & 
GETCLASS 

INITRA 

MOWSTA 

PERFEVOL 

PERT REX 

PERWORK 

SUP  AM 

GE"‘EAM  & 
CHEi'  TEAM 

GETGUYS  6 
LAZYFIRS 

MANANA  & 
C/.RYOVER 

SCO RET YM 

WHEN  DONE 


FACTORS  RESPONSIBLE  FOR  LOWERING 
THE  CONSTRUCT  VALIDITY  OF  SHIP  II 


attempts  to  schedule  work  on  a  regular  basis; 
work  is'  scheduled  irregularly  aboard  ship. 

attempt  to  assign  all  idle  men  to  division 
training  classes;  other  attendance  criteria 
are  probably  significant  aboard  ship. 

attempts  to  schedule  division  training  classes 
on  a  regular  basis;  classes  are  scheduled  ir¬ 
regularly  aboard  ship. 

does  not  produce  erroneous  failure  reports  for 
equipments  that  have  not  actually  failed. 

assigns  fixed  teams  to  evolutions;  assignments 
are  more  flexible  aboard  ship, 

will  not  allow  performance  of  exercise  if  re¬ 
quired  subsystems  are  not  rated  C-l;  require¬ 
ments  are  more  flexible  aboard  ship. 

does  not  assign  men  to  jobs  that  are  too  large 
to  complete;  jobs  may  be  partially  completed 
aboard  ship. 

computes  deferrals  on  a  strict  probability  basis 
various  factors  influence  deferrals  in  the  fleet 

use  rigid  rules  for  priorities;  priorities  are 
more  flexible  aboard  ship. 

use  rigid  rules  for  selecting  men  to  do  ship's 
work;  assignments  are  more  flexible  aboard  ship. 

compute  work  remaining  and  carryover  by  simple 
formula;  factors  affecting  work  carryover »are 
more  complex  aboard  ship. 

uses  unrealistic,  simplistic,  probabilistic 
scoring  system  instead  of  evaluating  various 
factors  that  contribute  to  type  commander's 
readiness  rating. 

allows  jobs  to  be  done  at  regular  times; 
scheduling  is  not  so  regular  aboard  ship. 
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The  general  procedure  followed  in  modeling 
ship  functions  was  to  develop  a  logic  flow  chart  of  the  pro¬ 
cesses  and  activities  taking  place  on  the  ship,  then  convert 
that  into  a  computer  program  flow  chart.  Programming,  key¬ 
punching  and  debugging  then  followed.  The  problems  of  model 
validity  and  verisimilitude  were  an  important  consideration 
in  this  process.  Since  some  of  the  intended  model  uses  pre¬ 
cluded  empirical  validation  of  results,  particular  attention 
was  devoted  to  ensuring  the  correspondence  of  program  logic 
to  real  world  processes.  Operational  fleet  personnel  were 
consulted  extensively  during  functional  design.  Compromises 
sometimes  had  to  be  made  between  construct  validity  of  the 
model  and  programming  limitations.  In  cases  where  artifacts 
were  introduced  to  simplify  programming,  they  were  constructed 
so  as  not  to  introduce  non-correspondence  with  the  real  world. 
Real  world  processes  not  considered  relevant  to  the  model 
outputs  were  excluded. 

After  the  original  version  of  Ship  II  was  de¬ 
veloped,  an  extensive  review  of  the  model  was  conducted  under 
the  supervision  of  the  Simulation  Advisory  Group.  Comments 
were  solicited  from  Navy  personnel  representing  both  model 
users  and  model  developers .  The  comments  resulting  from  this 
review  were  responded  to  by  a  revision  of  the  model.  The  re¬ 
sulting  model,  which  is  the  current  version  of  Ship  II,  is 
more  closely  reflective  of  the  real  Navy  world  than  the  pre¬ 
vious  version. 

In  summary,  the  close  participation  of  Navy 
experts  in  the  development  and  revision  of  Ship  II  functional 
definition  provides  support  .for  believing  that  the  construct 
validity  is  satisfactory. 


Face  Validity 


A  summary  of  face  validity  judgements  for  Ship  II 
is  contained  in  Tables  4  through  10.  The  data  in  these  tables 
are  taken  from  the  studies  summarized  in  Section  II.  The 
judgements  of  whether  or  not  the  changes  in  the  dependent  vari¬ 
ables  appear  to  be  realistic  has  been  made  by  the  authors  of 
the  individual  studies  rather  than  by  the  author  of  this 
report.  Generally,  the  validity  judgements  are  based  on  the 
direction  of  change  in  the  dependent  variable  values.  No 
attempt  was  made  to  judge  the  correctness  of  the  magnitude 
of  the  change  in  dependent  variables  or  of  the  form  of  the 
curve  showing  changes  in  dependent  variables  as  a  function 
of  changes  in  independent  variables. 
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Each  dependent  variable  in  the  table  is  followed 
by  a  number  in  parenthesis.  This  number  corresponds  to  one 
of  the  nine  types  of  output  reports  discussed  in  Appendix  B, 
Section  B.  For  example,  if  a  dependent  variable  is  followed 
by  (5) ,  a  definition  of  that  dependent  variable  will  be  found 
in  section  B-5  of  Appendix  B.  In  addition,  each  table  will 
be  followed  by  a  brief  statement  of  the  rationale  behind  the 
validity  judgements  for  each  dependent  variable  in  the  table. 

C.  Reliability 

1 .  Discussion  of  Concept 


The  literature  reviewed  indicates  that  the  term  "reli¬ 
ability",  as  applied  to  stochastic  models,  includes  two  sep¬ 
arate  concepts.  The  first  deals  with  whether  different  users 
can  use  the  model  on  similar  problems  and  obtain  reasonable 
consistency  in  model  results.  When  a  model  is  applied  to  same 
or  similar  systems  by  various  users,  comparable  results  should 
T  be  obtained.  A  model  which  produces  contradictory  or  incon¬ 

sistent  results  when  applied  to  the  same  problem  by  different 
users  would  be  unreliable  and  the  results  from  the  model  could 
not  be  trusted. 

■  v 

The  second  viewpoint  on  reliability  is  that  a  model's 
reliability  is  simply  a  factor  which  determines  how  many  model 
runs  will  be  required  to  produce  a  given  level  of  output  pre¬ 
cision.  It  is  a  well  known  statistical  fact  that  large  samples 
f  l  produce  better  estimators  of  population  means  than  do  small 

samples.  In  addition,  confidence  intervals  become  shorter  as 
sample  size  increases.  As  the  number  of  simulation  runs  in¬ 
creases,  then,  the  resulting  means  values  of  output  variables 
will  become  increasingly  better  estimators  of  the  true  value 
of  these  variables.  The  lower  the  reliability  of  model  out¬ 
puts,  the  more  computer  runs  will  be  needed  to  achieve  any 
specified  level  of  precision  of  estimation.  Thus,  model  out¬ 
put  reliability  directly  affects  the  cost  of  using  a  model, 

;  when  the  criteria  for  number  of  replications  is  achievement 

of  a  given  level  of  precision  of  estimation.  # 

2 .  Ship  II  Reliability 

\  ? 

The  previous  discussion  has  presented  two  elements  in 
model  reliability.  It  wo  ild  be  desirable  to  describe  Ship  II 
in  terms  of  both  elements  of  reliability.  But  data  exist  only 
for  output  variance  of  certain  outputs.  With  regard  to  the 
consistency  criterion,  Ship  II  has  been  used  only  by  its 
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TABLE  4:  SUMMARY  OF  VALIDITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  REDUCED  .tANNING-TOTAL  SHIP- 


DEPENDENT  VARIABLE 


Percent  of  NWW(l) 
Watchstanders 
Non-Watchstanders 

Planned  maint. 
hrs  (1) 
Watchstanders 
Non-Watchstanders 

Corr.  maint.  hrs(l) 
Watchstnaders 
Non-Watchstanders 

Maint.  pers.- 
Q  hrs  (2) 

PM-No.  cancelled  (5) 

FM  &  S/A-hrs 
left  (6) 

FM  &  S/A-  intrpts(6) 

Div.  trng.- 
hrs/wk  ( 7 ) 

Pent  time  at  C-l 
(tot  ship)  (9) 

Tblsht  Q  hrs  (4) 

Repair  Q  hrs  (4) 

Trng.  read.-  hrs 
perf/hrs  sched  (8) 


DATA  IMPACT 


Not  Sup¬ 
port  Model 


Incon¬ 

clusive 


Direction  o 
Change  in  Dep. 
Var.  as 
Manning  Deer. 


Kim 


TABLE  4:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 

1.  Percent  standard  Navy  Workweek. 

Reduction  in  manning  would  be  expected  to  increase  workload 
for  the  remaining  men.  The  behavior  of  Ship  II  in  this 
respect  provides  support  for  Model  validity. 

2.  Planned  maintenance  hours. 

The  expectation  is  that  with  a  constant  PM  load,  decreases 
in  men  available  for  PM  should  increase  PM  hours  for 
remaining  men.  This  finding  occurred  only  for  watchstanders . 
The  data  for  non-watchstanders  was  judged  not  to  support  the 
model . 

3.  Corrective  maintenance  hours. 

A  decrease  in  manning  should  increase  the  CM  hours  for 
remaining  men.  Such  an  increase  was  found  for  watchstanders 
but  not  for  non-watchstanders. 

4.  Queue  hours-  Maintenance  Personnel. 

This  variable  represents  queues  formed  because  maintenance 
personnel  of  the  specified  rate  rating  &  NEC  are  not  avail¬ 
able  when  needed.  As  number  of  maintenance  personnel  are 
decreased,  the  same  rate  of  equipment  failure  would  increase 
the  demand  on  the  remaining  men,  thus  increasing  maintenance 
queues . 

5.  Planned  maintenance  cancellations. 

6.  FM  &  S/A  hours  not  done. 

7.  FM  &  S/A  interrupts. 

8.  Division  training-hours/week. 

The  increasing  demand  for  CM,  as  indicated  for  the  formation 
of  CM  queues,  should  be  reflected  in  reduction  of  emphasis 
on  lower  priority  tasks.  Both  PM,  FM,  and  S/A  are  lower 
priority  tasks.  Accordingly,  the  increase  in  PM  cancella¬ 
tions  and  the  increase  in  FM  and  S/A  hours  of  work  not  done 
both  indicate  that  Ship  II  is  behaving  correctly.  However, 
it  was  also  found  that  FM  and  S/A  interrupts  decreased. 
According  to  the  authors  of  the  study,  interrupts  are  mainly 
for  watch  standing  and  scenario  events,  which  are  the  same 
for  each  Manning  Condition.  The  number  of  interrupts  wcjuld 
be  expected  to  be  the  same  for  each  condition.  The  decrease 
in  interrupts  was  attributed  to  decrease  in  number  of  jobs 
assigned.  The  decrease  in  hours  spent  in  division  training 
which  is  also  a  lower  priority  activity,  also  supports  the 
model. 

9.  Percent  time  at  C-l  readiness  status  (total  ship) . 

The  lack  of  effect  of  reducing  manning  on  readiness  status 
was  classified  as  inconclusive.  Without  data  on  the  effect 
of  reduced  manning  on  equipment  failures,  the  proper  response 
of  readiness  status  cannot  be  determined. 


\ 

\ 

10.  Troubleshooting  queue  hours. 

11.  Repair  queues  hours. 

These  variables  represent  time  waiting  for  troubleshooting 
and  repair  of  equipment  to  begin  after  a  troubleshooting  or 
repair  requirement  has  been  identified.  The  expected  effect 
of  reduced  manning  on  these  queues  would  be  an  increase  in 
availability  of  maintenance  personnel.  This  finding  occurred 
for  troubleshooting  but  not  for  irepair. 

12.  Readiness  training  exercises. 

The  expected  effect  of  reducing  personnel  on  performance  of 
training  exercises  is  not  clear.  This  variable  was  classi¬ 
fied  as  inconclusive. 


TABLE  5:  SUMMARY  OF  VALIDITY  JUDGEMENTS 
INDEPENDENT  VARIABLE:  SCENARIO  LOAD6 


DEPENDENT  VARIABLE 

I 

|  DATA  IMPACT 

Direction  of 
Change  in  Dep. 
Var.  as  Seen. 
Load  Incr. 

Support 

Model 

Not  Sup¬ 
port  Model 

Incon¬ 

clusive 

! 

|  Percent  of  NWW  (1) 

Watchstander 

X 

Incr 

Non-Watchstander 

X 

Incr 

Total  Workload 

Total  division 

X 

Incr 

Watchstanders 

X 

Incr 

Non-Watchstanders 

X 

Incr 

Watch  Hrs  (1) 

Total  division 

X 

Incr 

Watchstanders 

X 

Incr 

Non-Watchstanders 

X 

i 

Incr 

Div.  Trng.  Hrs  (1) 

Total  division 

X 

Incr 

Watchstanders 

X 

Incr 

Non-Watchstanders 

X 

Incr 

Evolution  Hrs  (1) 

* 

Total  division 

X 

Deer 

Watchstanders 

X 

Deer 

Non-Watchstander 

X 

Incr 

Planned  maint. 

Hrs  (1) 

Total  division 

X 

Watchstander 

X 

Incr* 

Non-Watchstander 

X 

None 

Corr.  maint. 

Hrs  (1) 

Total  division 

.1 

X 

None 

Watchstander 

X 

Incr 

Non-Watchstander 

_ 1 

1 

« 

_ L 

X 

None 

0 Study  C,  Chapter  II 


40 


TABLE  5:  CONTINUED  SUMMARY  OF  VALIDITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  SCENARIO  LOAD 


DEPENDENT  VARIABLE 

1  DATA  IMPACT 

Direction  of 
Change  in  Dep 
Var.  as  Seen. 
Load  Incr. 

Support 

Model 

Not  Sup¬ 
port  Model 

Incon¬ 

clusive 

Fac.  Maint.  Hrs  (1) 

Total  division 

X 

None 

Watchstander 

X 

Deer 

Non-Watchstander 

X 

Incr 

Supp/Admin  Hrs  (1) 

Total  division 

X 

Deer 

Watchstanders 

X 

Incr 

Non-Watchstander 

X 

Deer 

PM-  No.  Sched  (5) 

X 

None 

PM-  total  manhours 

(5) 

X 

None 

FM  &  S/A  Hourse 

left  (6) 

X 

Incr 

PM  &  S/A  Intrpts  (6 

!  X 

Incr 

Div.  Trng.- 

No  sessions  (7) 

X 

Incr 

Div .  Trng . - 

Attendance  (7) 

X 

• 

Deer 

Div.  Trng.- 

Duration  (7) 

X 

Deer 

TABLE  5:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 


Percent  standard  Navy  workweek. 

Total  Workload. 

Increasing  total  external  workload  imposed  on  the  crew  should 
be  reflected  in  increased  work  hours .  The  fact  that  both  per 
cent  of  SNWW  and  total  workload  increased  provided  support 
for  model  validity. 


3 .  Watch  hours . 

The  increasing  Scenario  load  is  accomplished  by  increasing 
the  hours  during  which  the  ship  is  on  Condition  III  and 
Condition  I  watch.  The  observed  increases  in  watch  hours 
stood  by  the  crew  indicates  the  increasing  the  Scenario 

load  is  having  the  proper  effect. 

/ 

4.  Division  training  hours. 

Since  division  training  is  a  lower  priority  activity,  no 
reason  was  identified  why  division  training  hours  should 
increase  as  the  ship  moves  from  Condition  iv  to  III,  to  I. 
If  there  was  any  effect  at  all,  a  decrease  in  hours  would 
have  been  expected. 

5.  Evolution  hours. 

The  findings  for  this  variable  are  somewhat  difficult  to 
interpret.  Evolution  is  the  highest  priority  in  the  model 
and  always  gets  accomplished  when  scheduled.  This  reflects 
the  priority  given  to  evolutions  aboard  ships.  Therefore 
the  evolution  hours  for  the  division  should  have  remained 
approximately  constant,  but  the  data  show  a  decrease  in 
evolution  hours.  The  findings  for  watchstanders  and  non- 
watchstanders  are  judged  to  be  inconclusive. 

6.  Planned  maintenance  hours. 

The  findings  for  PM  hours  are  confusing.  Increasing 
scenario  load  should  result  in  decrease  in  PM  hours,  or  at 
least  no  change.  But  PM  hours  for  watchstanders  .increased, 
a  trend  judged  to  be  inconsistent  with  reality. 

7.  Corrective  maintenance  hours. 

Increasing  watch  condition  from  IV  to  III  to  I  increases 
equipment  operating  hours  and  thus  equipment  failures  would 
also  increase.  Whether  these  failures  would  be  reflected 
in  increased  CM  manhours  is  difficult  to  determine.  There¬ 
fore  the  data  for  this  variable  have  been  classified  as 
inconclusive. 

8 .  FM  hours . 

FM  is  a  low  priority  task.  FM  hours  would  be  expected  to 
decrease  as  scenario  load  increases.  However  since  A 

increasing  the  scenario  load  is  actually  an  increase  in 
watch  hour  requirements,  a  differential  for  watchstanders 
and  non-watchstanders  would  be  expected.  In  Condition  II, 
for  example,  FM  performed  by  watchstanders  would  decrease 
but  would  increase  for  non-watchstanders.  The  data  reflect 
this  type  of  differential  effect. 
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9.  S/A  hours. 

The  findings  for  this  variable  are  inconsistent  with  the 
findings  from  FM  hours/week.  S/A  hours  for  watchstanders 
increased  as  scenario  load  increased,  a  finding  judged  not 
to  support  the  model.  S/A  hours/week  decreased  for  non- 
watchstanuers  and  the  total  division.  This  finding  is 
certainly  reasonable  but  confusing  when  compared  with  the 
FM  data.  Therefore  the  data  for  non-watchstanders  has  been 
judged  inconclusive. 

10.  Number  of  PM  jobs  scheduled. 

The  number  of  PM  jobs  scheduled  on  board  a  ship  would  cer¬ 
tainly  decrease  during  Condition  I  watch.  The  model  showed 
no  such  change. 

11.  Number  of  PM  jobs  cancelled. 

The  finding  that  PM  cancellations  increase  is  consistent 
with  reality. 

12.  PM  total  manhours. 

The  finding  for  this  variable  corresponds  to  variable  10- 
number  of  jobs  scheduled.  Ship  II  does  not  decrease  PM 
during  Condition  I  as  would  be  expected  on  board  a  ship. 

13.  FM  and  S/A  hours  not  done. 

14.  FM  and  S/A  interrupts. 

FM  and  S/A  are  the  lowest  priority  tasks  aboard  ship. 

Both  hours  of  work  not  done  and  interrupts  would  increase 
during  Condtion  III  and  I. 

15.  Division  training  -  number  of  sessions. 

16.  Division  training  -  attendance. 

17.  Division  training  -  duration. 

Division  training  is  also  a  relatively  low  priority  activity. 
The  decrease  in  attendance  and  duration  as  scenario  load 
increases  is  realistic  but  the  increase  in  number  of  train¬ 
ing  sessions  is  not. 
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TABLE  6:  SUMMARY  OF  VALIDITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  TIME  IN  FUNCTION7 


,  DATA  IMPACT  Direction  of 


Support 

Not  Sup- 

Incon- 

Change  in  Dep. 

DEPENDENT  VARIABLE 

Model 

port  Model 

elusive 

Var.  as  TIF 
decreases 

Corr.  maint.  hrs(l) 
Planned  maint. 

X 

None 

hrs  (1) 

Maint.  pers- 

X 

None 

Q  hrs  (?) 

Pent  time  at  C-l 

X 

None 

(Aux.  eng.)  (9) 

Pent  time  at  C-l 

X 

None 

(AAW  sch  &  trk)  (9) 

X 

None 

Tblsht  Q  hrs  (4) 

X 

None 

Repair  Q  hrs  (4) 

PM-  No. 

X 

None 

cancelled  (5) 

FM  &  S/A-  hrs 

X 

None 

left  (6) 

FM  &  S/A 

X 

None 

intrpts  (6) 

X 

None 

Div.  trng.-  hrs 
perf/hrs  sched(7) 
Trng.  read.-  hrs 

X 

None 

perf/hrs  sched(8) 

X 

None 
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TABLE  6:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 

1.  Corrective  maintenance  hours. 

This  variable  has  been  classified  as  not  supporting  Ship  II, 
but  the  interpretation  of  this  variable  is  complex.  As  equip¬ 
ment  failures  increase,  the  hours  spent  in  repair  and  trouble¬ 
shooting  should  also  have  increased.  The  result  was  not  sta¬ 
tistically  significant,  but  examination  of  the  curves  shows 
that  a  significant  difference  may  be  present  on  the  last  three 
weeks  (8,  9,  10).  The  smallest  difference  between  baseline  and 
reduced  TIF  runs  is  180,  120  and  200  hours  for  weeks  8,  9,  and 
10.  The  statistical  significance  of  these  differences  cannot 
be  determined  from  data  reported  in  the  study.  No  discussion 
of  the  rest  of  the  dependent  variables  will  be  included.  The 
relationship  between  TIF  and  these  variables  is  not  clear  enough 
to  judge  whether  the  results  support  or  do  not  support  model 
validity. 


TABLE  7;  SUMMARY  OF  VALIDITY  JUDGEMENTS 
INDEPENDENT  VARIABLE:  TIME  BETWEEN  EQUIPMENT  FAILURE8 


- 

DATA  IMPACT 

Direction  of 

DEPENDENT  VARIABLE 

Support 

Model 

Not  Sup¬ 
port  Model 

Incon¬ 

clusive 

Change  in  Dep 
Var.  as  MTBF 

Incr . 

Corr.  maint. 
hrs  (1) 

Planned  maint. 

hrs  (1) 

Maint.  pers- 
Q  hrs  (2) 

Pent  time  at  C-l 
Aux.  eng.  (9) 

Pent  time  at  C-l 
(AAW  sch  &  trk)  (9] 

Tblsht  Q  hrs  (4) 
Repair  Q  hrs  (4) 
PM-No.  cancelled (5) 
FM  &  S/A-hrs 
left  (6) 

FM  &  S/A-  hrs 
left  (6) 

Div,  trng.-  hrs 
perf/hrs  sched(7) 
Trng.  read.-  hrs 
perf/hrs  sched(7) 


Study  B,  Chapter  II 


TABLE  7:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 


1.  Corrective  maintenance  hours. 

Increases  in  MTBF  should  be  reflected  primarily  in  higher 
equipment  failures.  Corrective  maintenance  hours  should 
correspondingly  increase.  No  such  effect  was  found.  MTBF 
was  not  greatly  varied  from  its  baseline  value  (value  of 
1.2  BL,  1.4  BL  and  1.6  BL  were  used).  However,  it  was 
varied  enough  to  cause  decreases  in  equipment  failure.  The 
fact  that  corrective  maintenance  hours  did  not  correspond¬ 
ingly  increase  indicates  a  potential  problem  in  the  model. 

2.  Planned  maintenance  hours. 

The  implications  of  this  variable  are  inconclusive.  There 
is  no  clear  cut  reason  why  changes  in  MTBF  would  be  expected 
to  have  any  effect  on  planned  maintenance  hours. 

3.  Maintenance  personnel  queue  hours. 

Increases  in  equipment  failures  might  be  expected  to  cause 
increases  in  queues.  But  certain  increases  can  be  handled 
by  increasing  workload  without  a  need  for  queues.  The  data 
from  the  study  are  not  clear  whether  the  equipment  failures 
were  large  enough  to  be  expected  to  result  in  queues. 

4.  Percent  of  time  at  C-l  (Aux.  eng.) 

5.  Percent  of  time  at  C-l  (AAW  search  &  track) . 

The  data  from  the  study  were  not  sufficient  to  determine  if 
the  increased  equipment  failures  were  large  enough  to  expect 
a  degredation  in  equipment  readiness. 

6.  Troubleshooting  queues-  total  ship. 

7.  Repair  queues  -  total  ship. 

The  comments  for  variable  3  apply  here..  Whether  the  equip¬ 
ment  failures  increased  enough  to  cause  troubleshooting  and 
repair  queues  is  not  apparent  from  the  data. 

8.  Planned  maintenance  cancellations. 

The  relationship  between  decreases  in  MTBF  and  changes  in 
PM  cancellations  is  obscure.  Such  a-  relationship  may  be 
present  but  lack  of  knowledge  concerning  its  form  dictated 
an  "inconclusive"  judgement. 

9.  FM  &  S/A  work  not  done. 

10.  FM  &  S/A  interrupts. 

11.  Division  training. 

All  of  these  variables  have  been  classed  as  inconclusive  be¬ 
cause  of  the  lack  of  knowledge  about  how  increases  in  MTBF 
would  be  expected  to  effect  these  variables. 
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TABLE  8;  SUMMARY  OF  VALIDITY  JUDGEMENTS 
INDEPENDENT  VARIABLE;  NO.  OF  STG  2&39 


DATA  IMPACT"  !  Direction  of 


DEPENDENT  VARIABLE 

Support 

Not  Sup- 

Incon- 

Change  in  Dep 
Var.  as  STG's 

Model 

port  Model 

elusive 

decrease 

Watch  Hrs  (1) 

X 

Incr 

Percent  of  NWW  (1) 
Maint.  pers.-Q 

X 

Incr 

Hrs  (2) 

X 

None 

Maint.  pers.- 
No.  Qs  (2) 

Maint.  pers.- 

X 

None 

No.  intrpts  (2) 

X 

None 

Equip.  Avail,  pet (4) 
Tblsht  &  Repair 

X 

Variable 

Q  Hrs  (4) 

X 

None 

PM-No.  delays  (5) 

X 

Incr 

PM-No.  cancelled  (5) 
FM  &  S/A-  hrs 

X 

Incr 

left  (6) 

X 

Incr 

FM  &  S/A- 
intrpts  (6) 

Trng.  read.- 

X 

None 

cancelled (8) 

Trng.  read.- 

X 

None 

delays  (8) 

Trng.  read.- 

i 

1 

X 

None 

delay  hrs  (8) 

X 

None 
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1.  Watch  hours. 

These  findings  seem  to  be  consisted  with  what  would  be  expected 
to  occur  on  a  real  ship.  As  number  of  men  available  decreases, 
mean  watch  hours  for  each  remaining  man  qualified  to  stand 
watches  would  be  exepcted  to  increase. 


2.  Percent  of  Standard  Navy  Workweek. 

When  amount  of  work  required  to  be  done  is  constant,  it  is 
reasonable  that  decrease  in  total  men  available  to  do  the 
work  would  result  in  a  higher  workload  for  the  remaining  men. 


3.  Maintenance  personnel  queues. 

4.  Maintenance  personnel  queue  hours. 

5.  Maintenance  personnel  interrupts 

The  values  for  these  dependents  variables  were  zero  for  all 
values  of  size  of  target  group  and  workload  level.  The 
implications  of  this  lack  of  response  on  model  validity  are 
not  clear.  It  may  be  that  equipment  failures  were  infrequent 
enough  that  2  men  could  handle  the  corrective  maintenance 
load  without  queues  or  interrupts  occurring.  But  no  data  is 
available  to  verify  this. 


6.  Mean  Equipment  Availability  Percent. 

The  results  obtained  from  the  simulation  indicated  that  when 
the  size  of  the  target  group  decreased,  equipment  availability 
increased  until  target  group  dropped  to  2;  availability  per¬ 
cent  then  dropped  to  its  lowest  point.  This  finding  does  not 
support  model  validity. 


7.  Troubleshooting  and  Repair  Queues. 

No  significant  effect  on  this  dependent  variable  was  obtained. 
This  finding  was  judged  as  inconclusive.  At  first  glance  it 
might  appear  that  decrease  in  personnel  available  for  mainten¬ 
ance  would  certainly  increase  the  number  of  troubleshooting 
and  repair  queuces  on  equipment  maintained  by  the  reduced 
number  of  personnel.  However,  with  all  personnel  present,  the 
highest  number  of  queues  was  0.022.  This  value  is  so  low 
that  perhaps  it  should  not  be  expected  to  decrease  as  mainten¬ 
ance  personnel  decreases. 


8.  Planned  Maintenance  Delays. 

9.  Planned  Maintenance  Cancellations. 

Both  of  these  variables  increased  as  group  size  decreased. 
Decrease  in  maintenance  men  from  6  to  2  would  be  expected  to 
have  this  effect  on  planned  maintenance  tasks. 
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10.  FM  and  S/A  hours  undone. 

As  number  of  personnel  available  to  do  the  work  decreased, 
the  hours  of  work  not  done  increased.  This  finding  is 
consistent  with  expectations. 

11.  FM  and  S/A  interrupts. 

This  variable  was  not  affected  by  decreases  in  personnel 
available.  This  finding  was  categorized  as  inconclusive 
because  it  was  not  clear  what  effect  would  be  found  on¬ 
board  ship.  FM  and  3/A  interrupts  occur  when  a  higher 
priority  work  requirement  exists.  Whether  or  not  decreases 
in  personnel  should  have  increased  higher  priority  work  re¬ 
quirements  and  therefore  increased  interrupts  were  not 
clear. 

12.  Mean  Cancellations  (Training  Readiness) . 

13.  Mean  Delays  (Training  Readiness) . 

14.  Mean  Delay  Hours  (Training  Readiness). 

The  study  data  showed  that  these  variables  were  not  effected 
by  changes  in  group  size.  Delays  and  cancellations  in  training 
readiness  exercises,  occur  when  required  equipment  is  not  oper¬ 
ating.  Whether  decreases  in  STG's  from  6  to  2  would  actually 
cause  large  enough  equipment  down  time  to  cancel  delay  and  exer¬ 
cises  on  a  real  ship  was  not  clear. 


TABLE  9:  SUMMARY  OF  VALIDITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  LEVEL  OF  FM  &  S/A  WORKLOAD10 


- 

DATA  IMPACT 

Direction  of 

DEPENDENT  VARIABLE 

Support 

Model 

Not  Sup¬ 
port  Model  ; 

Incon¬ 
clusive  j 

_ 1 

Change  in  Dep. 
Var.  as  Work¬ 
load  decreases 

Watch  hrs  (1) 

X 

None 

Percent  of  NWW  (1) 

X 

None 

Maint.  pers.- 

No.  Qs  (2) 

X 

None 

Maint.  pers.- 

Q  hrs  (2) 

X 

None 

Maint.  pers.- 

:  ;  i 

No.  intrpts  (2) 

i  x  i  i 

None 

Equip,  avail. 

i 

i 

1 

i 

i 

pet.  (4) 

X 

Deer 

Tblsht.  &  repair 

i 

' 

1 

Q  hrs  (4) 

X 

None 

PM-  No.  delays (5) 

X  :  i 

None 

PM-  No. 

1 

cancelled  (5) 

X 

i  None 

FM  &  S/A-  hrs 

1 

left  (6) 

X 

| 

1 

, 

Deer 

FM  &  S/A- 

1 

■ 

intrpts  (6) 

X 

, 

None 

Trng.  read. - 

1  • 

« 

cancelled  (8) 

X 

None 

Trng.  read.- 

delays  (8) 

X 

None 

Trng.  read.- 

delays  hrs (8) 

X 

None 

t'  r  -£ 
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TABLE  9:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 

1.  Watch  Hours. 

Since  watch  station  manning  takes  priority  over  all  other 
tasks,  except  evolution,  a  decrease  in  FM  &  S/A  workload 
from  100%  of  normal  to  20%  of  normal  would  be  expected  to 
have  no  effect  on  the-  total  number  of  watch  hours  worked. 

2.  Percent  of  Standard  Navy  Workweek. 

The  finding  that  %  SNWW  decreases  as  workload  imposed  on 
the  men  decreases  is  entirely  consistent  with  what  would 
be  expected  to  occur  on  board  a  ship. 

3.  Maintenance  personnel  queues 

4.  Maintenance  personnel  queue  hours 

5.  Maintenance  personnel  interrupts 

The  relationship  between  FM  &  S/A  workload  and  maintenance 
personnel  performance  is  remote  and  therefore  variation  in 
the  workload  would  be  expected  to  have  little  or  no  influence 
on  these  3  dependent  variables. 

6.  Mean  Equipment  Availability  Percent. 

Equipment  availability  percent  decreased  as  workload  decreased. 
This  finding  is  inconsistent  with  the  expectation;  certainly 
decrease  in  FM  workload  would  not  result  in  decreases  in  equip¬ 
ment  readiness  aboard  a  ship. 

7.  Number  of  troubleshooting  and  repair  queues. 

FM  &  S/A  workload  would  be  expected  to  have  no  effect  on  this 
dependent  variable.  This  finding  is  consistent  with  reality. 

8.  Planned  maintenance  delays. 

9.  Planned  maintenance  cancellations. 

Both  of  these  variables  are  only  remotely  related  to  FM  & 

S/A  workload.  The  finding  that  decreases  in  workload  had  no 
effect  on  these  variables  is  reasonable. 

10.  FM  &  S/A  hours  undone. 

It  is  obvious  that  as  external  FM  &-S/A  workload  decreases 
the  hours  of  such  work  not  accomplished  should  also  decrease. 

11.  FM  &  S/A  interrupts.  ' 

The  finding  that  decreases  in  FM  &  S/A  workload  resulted  in 
decreases  in  FM  &  S/A  interrupts  is  obviuosly  reasonable. 

If  interrupts  are  zero  at  100%  workload,  then  reductions 

in  workload  can  result  in  no  further  decreases  in  interrupts. 

12.  Cancellations  (Training  Readiness). 

13.  Delays  (Training  Readiness) . 

14.  Delay  hours  (Training  Readiness) . 

These  3  variables  have  been  classed  as  supporting  the  model 
because  it  would  be  expected  that  decreases  in  FM  &  S/A 
workload  would  have  no  effect  on  training  readiness. 
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TABLE  10;  SUMMARY  OF  VALIDITY  JUDGEMENTS 
INDEPENDENT  VARIABLE:  TASK  REDUCTION  &  MANPOWER  REDUCTION 


DATA  IMPACT 

birection  oi 

DEPENDENT  VARIABLE 

Support 

Model 

Not  Sup¬ 
port  Model 

Incon¬ 

clusive 

Change  in  Dep. 
Var.  as  Work- 
Load  decreases 

Total  Workload  (1) 

X 

Deer 

Percent  of  NWW  (1) 

X 

Deer 

Direction  of 
Change  in  Dep. 
Var.  as  Man¬ 
ning  decreases 

Total  Workload  (1) 

X 

Incr 

Percent  of  NWW  (1) 

X 

Incr 

TABLE  10:  RATIONALE  FOR  VALIDITY  JUDGEMENTS 

1.  Mean  hours  worked. 

2.  Percent  of  SNWW. 

Both  of  these  variables  decreased  when  the  64  hours  of  ships 
work  was  removed  from  the  ship.  This  finding  clearly  sup¬ 
ports  Ship  II  validity. 

3.  Mean  hours  worked. 

4.  Percent  of  SNWW 

After  eight  men  were  removed,  these  dependent  variables 
increased  in  value,  a  result  which  would  be  expected  to 
occur  aboard  a  ship. 


developers  and  on  a  non-overlapping  set  of  problems.  Little 
data  on  consistency  is  therefore  available  from  examination  of 
Ship  II  studies.  However,  very  little  of  Ship  II  input  data 
is  based  on  subjective  estimates  of  the  users.  Therefore,  the 
results  obtained  with  Ship  II  would  be  expected  to  show  ade¬ 
quate  consistency  across  users  but  no  data  exist  to  show  that 
this  is  the  case.  The  data  on  reliability  of  Ship  II  outputs 
was  reported  by  Schwartz,  Parker  £  Rhodes  (1970).  The  data 
from  this  study  will  be  presented  without  comment  because  it 
is  difficult  to  determine  the  implications  of  this  data  for 
model  quality.  However,  this  study  does  sho w  that  data  aggre¬ 
gated  at  the  total  ship  level  is  more  reliable  and  fewer  num¬ 
ber  of  replications  will  be  necessary  to  achieve  any  given 
level  of  precision  than  if  division  level  data  is  used. 

The  method  used  for  calculation  of  reliability  values 
is  as  follows: 

a.  All  output  variables  studies  were  listed  (Column 
1,  Table  11) . 

b.  Measurement  units  and  levels  of  treatment  were 
assigned  (Column  2,  table  11). 

c.  The  pooled  mean  value  of  the  statistic  was  com¬ 
puted  for  all  relevant  model  runs  (Column  3) .  The  pooled 
mean  value,  x,  is  equal  to  the  sum  of  the  means  divided  by 
the  number  of  runs,  i.e.,  x  =£xj/N. 

d.  The  pooled  standard  deviation  of  the  statistic 
was  then  computed  (Column  4),  in  the  following  fashion: 

s  =  /£  S.2/N  ; 

where  s  is  the  pooled  standard  deviation, 

s^2  is  the  sample  variance,  and 
N  is  the  number  of  samples  • 

e.  Since  the  primary  concern  was  the  probability  that 
results  would  be  repeated  within  10  and  20  percent  of  the 
expected  value  (x) ,  confidence  bandwidth  of  +.10  and  ±.20  were 
selected  in  this  reliability  evaluation  (Columns  5  and  6) . 

For  each  output  variable  treated  in  the  test  runs,  the  mean 
and  standard  deviations  were  used  to  compute  the  probability 


53 


that  the  measured  output  would  fall  within  the  conficence 
bandwidth.  The  method  of  computation  was: 

R[cl  =0  (u)-ff(-u), 

where 

u  =  cx  , 
s 

R[c J  =  Reliability  value 

0  =  Fractile  of  the  normal  distribution 

x  =  Pooled  mean 

s  =  Pooled  standard  deviation 

c  =  Relative  confidence  bandwidth 

This  computation  first  determines  the  number  of  standard  devi¬ 
ations  which  fall  between  ±  10%  (or  +  20%)  of  the  calculated 
mean.  The  percentage  of  the  normal  curve  which  falls  within 
this  number  of  standard  deviations  from  the  mean  is  then 
determined.  This  percentage  figure  is  the  reliability  indi¬ 
cator  which  appears  in  the  tables. 
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D.  Sensitivity 

1.  Discussion  of  Concept 


Sensitivity  can  be  defined  as  the  degree  to  which  the 
dependent  variables  in  a  model  are  responsive  to  changes  in 
values  of  independent  variables. 

If  a  model  is  to  be  useful  for  investigating  relation¬ 
ships  between  variables,  the  dependent  variables  must  respond 
properly  to  changes  in  the  values  of  the  independent  variables. 
One  of  the  major  problems  in  testing  sensitivity  is  determi¬ 
nation  of  what  this  proper  response  is.  Testing  of  model  sen¬ 
sitivity  therefore  requires  data  on  real  world  relationships 
among  variables.  To  illustrate  this  point,  consider  two  types 
of  real  world  relationships  among  independent  and  dependent 
variables:  (1)  the  real  world  relationship  between  variables 
is  high  positive  or  negative;  (2)  the  real  world  relationship 
is  small  or  zero.  Assume  that  a  sensitivity  test  has  been 
made  and  it  has  been  found  that  an  output  variable  does  not 
change  as  an  input  variable  is  changed.  If  the  first  relation¬ 
ship  holds,  then  the  model  shows  lack  of  sensitivity.  On  the 
other  hand,  if  relationship  2  holds,  then  the  model  is  validly 
representing  the  real  world  situation.  Without  knowing  which 
relationship  underlies  the  input-output  variables,  either 
situation  may  be  eaually  likely.  The  results  of  a  sensitivity 
test  can  be  properly  interpreted  only  when  data  on  actual  re- 
^  lationships  among  the  variables  is  available.  Simulation 

models  are  usually  used  where  problem  complexity  precludes 
either  analysis  or  empirical  study  of  the  problem.  The  type 
f  of  empirical  data  needed  to  properly  interpret  a  sensitivity 

j  analysis  may  be  difficult  to  obtain  simply  because  of  the 

nature  of  the  situations  to  which  simulation  models  are  applied. 

As  defined  in  this  paper,  sensitivity  represents  the 
degree  of  statistical  relationship  between  input  and  output 
values.  This  degree  of  relationship  is  indicated  by  the  pro¬ 
portion  of  output  variance  accounted  for  by  variation  in  in¬ 
put  values.  When  the  variance  of  an  output  variable  is  small, 
it  may  seem  appropriate  to  conclude  that  the  output  is  not 
sensitive  to  whatever  input  variable  was  varied.  However, 
it  is  entirely  possible  for  a  strong  degree  of  relationship 
between  input  and  output  variables  to  exist  when  output  vari¬ 
ance  is  small.  The  outDut  variable  would  indeed  be  sensitive 
to  variation  in  the  input  variable  even  though  output  variance 
is  small. 
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One  of  the  problems  with  this  definition  of  sensitivity 
is  choosing  the  appropriate  measure  of  sensitivity.  The  appro¬ 
priate  index  of  proportion  of  variance  in  output  accounted  for 
by  input  variation  is  *X/  (Hayes,  1963).  However,  this  statistic 
was  not  calculated  in  the  studies  reviewed  in  Section  II. 
Nevertheless,  the  presence  or  absence  of  a  statistical  rela¬ 
tionship  can  be  roughly  determined  if  the  data  were  subjected 
to  tests  of  significant  of  differences  between  independent 
variable  values.  The  occurrence  of  a  significant  value  for 
such  a  test  of  significance  guarantees  that  some  degree  of 
statistical  relationship  is  Dresent  in  the  data.  However,  it 
is  not  possible  to  estimate  the  strength  of  this  relationship 
from  the  results  of  the  test. 

The  problem  in  using  statistical  significance  as  the 
criteria  for  sensitivity  is  that  strong  degrees  of  statistical 
relationship  may  be  missed  by  this  criteria.  It  is  true  that 
a  significant  result  guarantees  that  some  (possibly  small) 
degree  of  relationship  exists  but  it  is  also  entirely  possible 
for  a  strong  degree  of  statistical  relationship  to  exist  in 
data  which  do  not  exhibit  a  significant  result.  The  factor 
which  determines  whether  a  significant  statistical  test  really 
represents  a  strong  statistical  relationship  is  sample  size. 

As  Hayes  (1963)  says:  "Virtually  any  study  can  be  made  to  show 
significant  results  if  one  uses  enough  subjects,  regardless 
of  how  nonsensical  the  result  may  be"  (P.  326) .  The  sample 
sizes  used  in  all  of  the  Ship  II  studies  have  been  small  (10) 
at  largest) ;  therefore  there  is  reason  to  feel  that  all  the 
significant  results  represent  at  least  a  moderate  degree  of 
statistical  association.  But  because  of  the  low  sample  size, 
it  is  not  unlikely  that  strong  statistical  relationships 
might  have  been  present  when  a  test  was  non-significant. 
Unfortunately,  the  studies  do  not  contain  the  data  needed 
to  clarify  the  situation.  Nevertheless,  the  sensitivity  in¬ 
formation  which  does  exist  will  be  summarized  even  though  it 
may  be  incomplete  and  difficult  to  interpret. 

2 .  Evaluation  of  Ship  II  Sensitivity 

A  summary  of  information  about  Ship  II  sensitivity, 
based  on  all  past  studies  of  Ship  II,  is  contained  in  this 
section.  The  indicator  of  the  presence  of  a  statistical 
relationship  is  a  significant  statistical  test.  With  one 
exception,  therefore,  these  tables  will  class  as  sensitive 
only  those  dependent  variables  which  show  a  statistically 
significant  relationship  with  the  independent  variable.  The 
one  exception  is  those  variables  for  which  a  "no  change"  re¬ 
lationship  was  judged  in  the  validity  tables  as  lending  support 


to  model  validity.  In  these  cases,  lack  of  significant;  rela¬ 
tionship  between  input  and  output  cannot  be  classed  as  insen¬ 
sitivity  because  such  a  relationship  reflects  the  kind  of  vari¬ 
ation  found  in  the  real  world.  Judgements  of  this  type  are 
classed  in  the  "non-applicable"  column  in  Tables  12  -  18. 

Each  table  summarizes  data  for  a  separate  independent  variable. 
The  dependent  variables  in  the  tables  are  the  same  as  in  the 
validity  tables  (tables  4  -  10).  As  noted  in  those  tables, 
the  number  in  parenthesis  afcer  each  dependent  variable  in 
tables  12  -  18  refers  to  the  paragraph  in  section  B  of 
Appendix  B  where  definition  of  that  variable  will  be  fourd. 

An  examination  o*  the  seven  sensitivity  tables  shows 
a  mixed  result.  In  three  studies,  the  majority  of  the  de¬ 
pendent  variables  were  found  to  be  sensitive  to  the  indepen¬ 
dent  variable  variation  (Tables  12,  13  &  18).  In  one  study 
the  results  were  about  evenly  divided  between  sensitive  and 
non-sensitive  (Table  16).  None  of  the  dependent  variables 
in  the  remaining  study  were  sensitive  to  either  of  two  inde¬ 
pendent  variables  (Tables  14  and  15).  The  question  of  in¬ 
terest  is  whether  the  lack  of  sensitivity  shown  in  some  of 
these  studies  is  a  characteristic  of  the  model  or  a  result 
of  the  studies.  With  regard  to  the  results  in  Tables  14  and 
15,  the  authors  of  that  study  (Study  B,  Chapter  II)  believe 
that  the  effect  of  MTBF  and  TIF  variation  were  masked  by  large 
equipment  downtimes  which  occurred  because  of  long  parts  de¬ 
ferrals.  For  some  equipments  readiness  was  degraded  to  the 
point  that  some  scheduled  scenario  events  could  not  take 
place.  It  is  reasonable  that  with  large  amounts  of  equip¬ 
ment  downtimes  occurring,  variation  in  MTBF  and  TIF  would 
not  be  significant.  This  possible  masking  effect  appears 
even  more  reasonable  when  considering  that  the  range  over 
which  MTBF  and  TIF  varied  was  small.  MTBF  varied  only  be¬ 
tween  baseline  value  and  1.6  BL;  TIF  varied  between  BL  and 
0.625  BL. 

One  other  factor  should  be  kept  in  mind  when  examin¬ 
ing  Tables  14,  15,  16  &  17.  The  first  part  of  this  section 
pointed  out  that  sample  size  determined  the  correspondence 
between  statistical  significance  and  degree  of  statistical 
relationship.  With  very  large  samples,  a  weak  degree  of 
relationship  could  be  present  even  though  the  statistical 
test  is  significant.  Conversely,  with  very  small  samples, 
a  strong  degree  of  relationship  could  be  present  even  when 
a  test  is  non-significant.  The  sample  size  for  these  two 
studies  were  smail  enough  that  a  real  possibility  exists  that 
the  latter  case  applies.  For  example.  Tables  16  and  17  were 
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TABLE  12:  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  REDUCED  MANNING-TOTAL  SHIP 


14 


DEPENDENT  VARIABLE 


jam 


Sensitive 


MPACT 


Not  Sensitive 


Non- 

Applicable 


Percent  of  NWW  (1) 
Watchstanders 
Non-Wa tchs  tander 


X 

X 


Planned  Maint. 
hrs  (1) 

Watchstanders  X 

Non-Watchstanders  X 


Corr.  maint.  hrs  (1) 
Watchstanders 
Non-Watchstanders 


X 


X 


Maint.  pers.- 
Q  hrs  (2) 

PM-No.  cancelled  (5) 
FM  &  S/A-hrs 
left  (6) 

FM  &  S/A-  intrpts  (6) 
Div.  trng.- 
hrs/wk  (7) 

Pent  time  at  C-l 
(tot  ship)  (9) 

Tblsht  Q  hrs  (4) 
Repair  Q  hrs  (4) 

Trng.  read. -hrs 
perf/hrs  sched  (8) 


x 

X 

X 

X 

X 


X 


X 

X 

X 
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TABLE  13:  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  SCENARIO  LOAD15 


DEPENDENT  VARIABLE 

DATA  IMPACT 

EBKBE 

Not  Sensitive 

Percent  of  NWW  (1) 

Watchstander 

X 

Non-Watchstander 

X 

Total  Workload 

Total  division 

X 

Watchstanders 

X 

Non-Watchstanders 

X 

Watch  Hrs  (1) 

Total  division 

X 

Watchstanders 

X 

Non-Watchstanders 

X 

Div.  Trng .  Hrs  (1) 

Total  division 

X 

Watchstanders 

X 

Non-Watchstanders 

X 

Evolution  Hrs  (1) 

Total  division 

X 

Watchstander 

X 

Non-Watchstander 

X 

Planned  Maint.  Hrs  (1) 

Total  division 

X 

Watchstander 

X 

Non-Watchstander 

X 

Corr.  Maint.  Hrs  (1) 

Total  division 

X 

Watchstander 

X 

Non-Watchstander 

X 

i 

Non-  \ 

Applicable 

_  '  j 


15 
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TABLE  13:  CONTINUED  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  SCENARIO  LOAD 


DEPENDENT  VARIABLE 

1 

DATA  IMPACT  | 

Non- 

Applicable 

Sensitive 

1 

Fac.  Maint.  Hrs  (1) 

Total  division 

X 

Watchstander 

X 

Non-Watchstander 

X 

Supn/Admin  Hrs  (1) 

io-tal  division 

X 

Watchstanders 

X 

Non-Watchstander 

X 

PM-  No.  Sched  (5) 

X 

PM-  Total  manhours  (5) 

X 

FM  &  S/A  Hours  left  (6) 

X 

FM  &  S/A  Intrpts  (6) 

X 

Div.  Trng.-  No 

sessions (7) 

X 

Div.  Trng.- 

Attendance  (7) 

X 

Div.  Trng.- 

* 

• 

Duration  (7) 

X 

« 
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TABLE  14:  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 
INDEPENDENT  VARIABLE:  TIME  IN  FUNCTION16 


DATA 

[MPACT 

Non- 

DEPENDENT  VARIABLE 

Sensitive 

Not  Sensitive 

Applicable 

Corr.  maint.  hrs(l) 
Planned  Maint. 

X 

hrs  (1) 

X 

X 

Maint.  pers- 
Q  hrs  (2) 

Pent  time  at  C-l 

X 

(Aux.  eng.)  (9) 

Pent  time  at  C-l 

X 

(AAW  sch  &  trk)  (9) 

X 

Tblsht  Q  hrs  (4) 

X 

Repair  Q  hrs  (4) 

PM-  No. 

X 

cancelled  (5) 

FM  &  S/A-  hrs 

X 

left  (6) 

FM  &  S/A 

X 

intrpts  (6) 

X 

Div.  Trng.-  hrs 

per f /hrs  sched(7) 

* 

X 

Trng.  read.-  hrs 

• 

perf/hrs  sched(8) 

X 

•^Study  B,  Chapter  II 
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TABLE  15;  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 
INDEPENDENT  VARIABLE:  TIME  BETWEEN  EQUIPMENT  FAILURE7 


DEPENDENT  VARIABLE 

DATA  IMPACT 

Sensitive  Not  Sensitive 

Non- 

Applicable 

Corr.  maint.  hrs  (1) 

X 

Planned  maint. 

hrs  (1) 

X 

Maint.  pers- 

Q  hrs  (2) 

X 

Pent  time  at  C-l 

Aux.  eng.  (9) 

X 

Pent  time  at  C-l 

(AAW  sch  &  trk)  (9) 

X 

Tblsht  Q  hrs  (4) 

X 

Repair  Q  hrs  (4) 

X 

PM-  No,  cancelled (5) 

X 

FM  &  S/A-  hrs 

left  (6) 

X 

FM  &  S/A-  hrs 

left  (6) 

X 

Div,  trng.-  hrs 

perf/hrs  sched(7) 

X 

Trng.  read.-  hrs 

perf/hrs  sched(7) 

X. 

■^Study  B,  Chapter  II 
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TABLE  16s  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 


INDEPENDENT  VARIABLE:  NO.  OF  STG  2&318 


i. 


DEPENDENT  VARIABLE 

DATA  IMPACT 

Applicable 

Sensitive  | 

I 

1 

Not  Sensitive 

Watch  Hrs.  (1) 

X 

Percent  of  NWW  (1) 

X 

Maint.  pers.-Q 

i 

Hrs  (2) 

! 

X 

Maint.  pers.- 

No.  Qs  (2) 

X 

Maint.  pers.- 

!  i 

No.  intrpts  (2) 

X 

Equip.  Avail,  pet (4) 

1 

X 

Tblsht  &  Repair 

1  ! 

Q  Hrs  (4) 

i  i 

X 

PM-No.  delays  (5) 

x 

PM-No.  cancelled  (5) 

S  x  i 

FM  &  S/A-  hrs 

| 

left  (6) 

x 

FM  &  S/A 

i 

intrpts  (6) 

X 

Trng.  read.- 

i 

cancelled  (8) 

i 

j 

X 

Trng.  read.- 

delays  (8) 

X 

Trng.  read.- 

1 

delay  hrs  (8) 

i 

X 

18 
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TABLE  17:  SUMMARY  OF  SENSITIVITY  JUDGEMENTS 


% 

is 

fe 

I 


INDEPENDENT  VARIABLE:  LEVEL  OF  FM  &  S/A  WORKLOAD19 


DEPENDENT  VARIABLE 


DATA  IMPACT 

Sensitive  Not  Sensitive 


Non- 

Applicable 


X 

Watch  hrs  (1) 

j, 

Percent  of  NWW  (1) 
Maint.  pore . - 

X 

No.  Qs  (2) 

X 

L 

y 
\  ■ 

ji' 

f- 

*: 

!:• 

1 

Maint.  pers.- 
Q  hrs  (2) 

Maint.  pers.- 

X 

No.  intrpts  (2) 

X 

Equip,  avail. 

* 

i 

pet.  (4) 

X 

f 

Tblsht.  &  repair 

1 

it 

Q  hrs  (4) 

X 

K. 

h 

PM-  No.  delays (5) 

fv 

PM-  No. 

X 

1 

cancelled  (5) 

FM  &  S/A-  hrs 

X 

E: 

»jf- 

left  (6) 

X 

;s; 

v’ 

FM  &  S/A- 

intrpts  (6) 

‘ 

• 

Trng.  read,- 

X 

fe 

cancelled  (8) 

1 

Trng .  rend . - 

X 

delays  (8) 

tf 

Trng .  read . - 

X 

1 

delays  hrs (8) 

X 

• 

si 

TABLE  18 
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based  on  a  sample  size  of  5  and  Tables  14  and  15  a  size  of  10. 

It  is  possible  that  the  results  in  Tables  14,  15,  16  &  17  do 
not  entirely  reflect  the  real  sensitivity  of  Ship  II. 

The  input-output  combinations  represented  in  these 
seven  tables  comprise  only  a  small  numher  of  the  total  number 
of  possible  Ship  II  input-output  combinations.  The  conclusion 
which  were  drawn  about  Ship  II  sensitivity  from  the  above 
studies  should  not  be  generalized  to  conclusions  about  the 
sensitivity  of  the  entire  model. 

E.  Utility 

The  basic  question  involved  in  evaluating  model  utility  is 
how  useful  is  the  model  in  aiding  in  the  solution  of  real 
problems  for  real  decision  makers.  Model  utility  must  be 
defined  by  specification  of  the  problem  areas  to  which  the 
model  may  potentially  be  applied,  mhis  section  will  contain 
a  discussion  of  the  problem  areas  to  which  Ship  II  can  be 
applied. 

Once  a  user  has  determined  the  models  which  are  generally 
applicable  to  his  problem,  the  selection  of  models  which  are 
specifically  appropriate  depends  mainly  on  examination  of  model 
outputs  and  inputs.  A  research  problem  is  defined  by  its 
impor _ant  independent  variables.  A  potential  user  must  there¬ 
fore  examine  Ship  II  independent  (input)  variables  to  deter¬ 
mine  if  Ship  II  can  be  applied  tc  his  particular  problem. 

A  user  must  also  examine  the  model  output  data  to  insure  that  the 
model  can  produce  the  data  he  needs  to  evaluate  his  tentative 
problem  solutions.  It  is  possible  for  a  model  to  have  the 
required  input  variables  to  study  a  problem  of  interest  but 
not  have  the  output  variables  which  the  user  requires  to 
assess  the  effect  of  his  independent  variables. 

A  distinction  between  two  different  types  of  applications 
of  a  model  is  useful  in  discussing  Ship  II  utility.  These 
two  types  of  applications  will  be  arbitrarily  called  type  A 
and  type  B  applications.  A  type  A  application  will  be  used 
to  refer  to  problems  whose  independent  variables  correspond 
directly  with  the  input  variables  of  the  model.  In  this  case, 
problem  formulation  is  simple:  the  user  selects  from  the  set 
of  model  input  variables  that  subset  which  is  identical  to 
the  independent  variables  of  his  problem. 

A  model  can  still  be  applied  even  if  the  oroblem  indepen¬ 
dent  variables  do  not  directly  correspond  to  a  subset  of  the 
model  input  variables,  but  an  additional  step  is  necessary: 
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the  user  must  determine  the  effect  of  changes  in  his  indepen¬ 
dent  variables  on  the  model  input  variables.  Ivhen  this  effect 
is  known,  the  further  steps  of  using  the  model  are  the  same 
as  in  a  type  A  application.  This  type  of  application  is  de¬ 
fined  as  a  type  B  application.  No  straightforward  method 
exists  for  determining  whether  a  given  problem  can  be  studied 
using  a  particular  model  by  a  type  B  application.  The  pro¬ 
cess  depends  mainly  on  the  skill  of  the  users. 

Determination  of  model  validity  for  type  B  applications 
is  complex.  The  validity  of  the  model  outputs  for  the  users 
independent  variables  (which  are  different  from  the  model  in¬ 
put  variables  in  a  type  B  application) ,  will  depend  upon  how 
accurately  the  user  specifies  the  relationship  between  his 
independent  variables  and  the  model  input  variables.  The 
quality  of  results  from  tvpe  B  applications  may  depend  as 
much  on  the  specification  of  this  relationship  as  it  does  on 
the  quality  of  the  model  itself. 

1.  Ship  II  Classes  of  Applications 


The  Ship  II  input  data  (independent  variables  for 
studies  using  Ship  II)  can  be  grouped  into  four  classes: 

•  Manning  information.  These  data  include  descrip¬ 
tions  of  billets,  lists  of  assigned  rates,  ratings, 
and  NECs,  etc. 

•  Equipment  information.  Data  in  this  class  consists 
of  such  items  as  failure  rates,  eauipment  identifi¬ 
cation  codes,  use  factors,  and  maintenance  times, 
etc . 

•  Task  and  training  information.  These  data  include 
such  items  as  task  identification  codes,  task  per¬ 
formance  times,  and  skill  type  and  level  required 
to  perform  the  task. 

•  Operations  requirements  information.  These  data  in¬ 
clude  lists  of  required  exercises  and  evolutions*, 
frequency  and  performance  times,  as  well  as  ship 
readiness  condition  (general  quarters,  etc.) 

These  four  types  of  data  define  the  four  general  clas¬ 
ses  of  studies  which  Ship  II  can  be  used  for.  Each  of  these 
four  will  be  discussed  in  more  detail,  but  no  attempt  will  be 
made  to  define  all  types  of  studies  to  which  Ship  II  can  be 
applied. 
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a.  Manning  Set  Studies 


The  model  allows  for  changes  to  be  made  in  the  ship¬ 
board  manning  set  six.  The  procedure  for  changing  manning  set 
is  not  overly  complicated.  In  addition  to  removing  and/or 
adding  input  data  cards  corresponding  to  the  personnel  to  be 
removed  and/or  added,  several  other  model  areas  may  need 
changes.  For  example,  a  new  manning  set  mav  reauire  changes 
in  watch  assignments,  overall  amount  of  work  assigned,  work 
responsibilities,  maintenance  crew  structure,  etc.,  depending 
upon  the  types  and  degree  of  personnel  restructuring. 

The  most  obvious  type  of  study  based  upon  change 
in  manning  set  is  to  study  the  effects  of  changina  the  gross 
numbers  of  personnel  on  the  ship,  or  within  certain  divisions, 
or  within  different  rating  groups.  In  conjunction  with  the 
changes  in  numbers  of  men  on  board,  the  personnel  skill  levels 
can  be  similarly  manipulated  to  assess  the  results  of  those 
changes  in  terms  cf  ship  operating  (output)  characteristics. 
Different  mixes  of  numbers  of  men  and  skills  types  can  be 
evaluated  to  try  to  achieve  a  specific  output  result  (e.g., 
specified  work  week  levels) ,  to  arrive  at  an  optimum  manning 
set,  or  to  investigate  degradations  resulting  from  different 
manning  nixes. 

Various  combinations  of  rate/rating/NEC  can  be 
used  to  study  changes  in  skill  levels  to  investigate  a  major¬ 
ity  of  skill  oriented  questions  of  interest  to  Navy  planners, 
such  as  readiness,  performance  of  maintenance,  training,  work¬ 
load  statistics,  errors,  delays,  etc.  An  example  application 
might  concern  the  effect  of  on-the-job  training  (OJT)  on  the 
workload  of  a  particular  ship’s  division.  *  The  OJT  condition 
could  be  demonstrated  by  allowing  some  of  the  lower  rate/ 
rating  combinations  to  perform  maintenance  on  selected  eauip- 
ment  they  were  not  previously  eligible  to  work  on,  thus  restruc¬ 
turing  the  breakdown  of  work  within  the  division,  to  see  what 
effects  that  would  have  on  the  overall  achievement  of  the 
division's  objectives. 

b.  Equipment  Set  Studies  • 

This  type  of  study  involves  making  changes  in  the 
model's  equipment  set.  Studies  involving  changes  in  the 
eeuipments  on  board  are  of  a  slightly  more  complicated  nature 
than  some  of  the  other  problem  areas.  Not  only  must  changes 
be  made  in  the  equipment  cards,  but  also  input  data  related 
lo  the  equipments  changed  must  be  added.  Operational  charac¬ 
teristics  of  the  equipments  to  be  added  must  be  analyzed  for 


the  eauipment  data  categories  that  the  model  considers,  e.g., 
deferral  probabilities,  MTRF,  MTTR,  PM,  impact  on  readiness, 
and  others.  In  addition  to  the  above  operational  character¬ 
istics,  the  eauipment  must  be  assessed  for  its  personnel  im¬ 
plications  and  impacts,  e.g.,  watch  requirements  operator/ 
repairman  levels,  integration  into  the  existing  manning  set, 
time  requirements  for  operation,  etc.  In  this  manner  the 
model  can  be  used  to  investigate  questions  concerning  the 
impact  of  automation,  design  changes,  and  other  eauipment 
oriented  innovations  upon  the  operation  of  the  modeled  ship, 
without  actually  having  to  produce  the  new  equipments  or  in¬ 
stall  them  on  board  the  ship. 

c.  Operational  Requirements  Studies 

A  change  in  the  deployment  status  of  the  ship 
model  requires  modification  of  many  categories  of  input  data 
as  well  as  changes  in  the  ship's  operational  scenario.  The 
input  data  categories  that  must  be  changed  are  those  related 
to  the  different  tempos  of  operations  associated  with  the 
different  ship  deployments.  The  ship  maintenance  require¬ 
ments,  ship  operation  and  mission  requirements,  failure 
rates,  training  schedules  and  readiness  exercises,  and  ships 
work  ,  all  depend  on  the  deployment  status  of  the  ship.  If 
the  questions  being  investigated  involve  long  periods  of  ship 
operation,  it  is  realistic  to  expect  the  ship  to  undergo 
several  changes  in  the  tempo  of  operations  due  to  changes  in 
deployment. 


d.  Task  Reauirement  Studies 


In  this  type  of  study,  the  external  task  load  im¬ 
posed  on  the  crew  is  varied.  This  load  can  be  varied  directly 
by  changes  in  the  workload  for  Facilities  Maintenance,  Sup¬ 
port  and  Administration,  and  Planned  Maintenance.  It  can  be 
effected  indirectly  by  changing  eauipment  MTBF,  Corrective 
Maintenance  times,  and  scenario  load  (which  effects  readiness 
condition,  number  of  readiness  training  exercises,  and  num¬ 
ber  of  evolutions) . 

In  the  simplest  study  of  this  type,  the  exterrfal 
task  load  is  varied  to  determine  the  effect  on  the  workload 
of  the  crew.  Often  the  question  of  interest  is  whether  any 
individuals,  NECs,  rates,  are  consistently  over  or  under 
worked.  A  natural  extension  of  this  type  of  study  when  cer¬ 
tain  personnel  are  under  worked  is  removal  of  these  personnel 
from  the  ship  after  assigning  their  remaining  tasks  to  other 
individuals.  The  model  is  then  run  with  both  reduced  tasks 
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and  reduced  manning  to  determine  if  the  remaining  personnel 
can  perform  the  reduced  task  set  (plus  all  other  work  require¬ 
ments  which  are  not  reduced,  such  as  watch  station  manning). 

2 .  Ship  II  Utility 

According  to  Siegel  (1973),  for  a  model  to  have  utility, 
it  must  provide  data  which  "are  not  possible  or  which  are  im¬ 
practical  to  otherv/ise  derive  or  which  cannot  be  derived  more 
economically"  (p.  9) . 

"There  are  two  aibauestions  subsumed  by  this  need  ful¬ 
fillment  question:  (1)  does  the  system  planner  need  the  type 
of  information  provided  by  Ship  II,  and  (2)  will  Ship  II  pro¬ 
vide  unique  data  and  accordingly  allow  the  decision  maker  to 
reach  the  appropriate  decision  at  a  higher  level  of  confidence? 
There  are  no  data  available  to  provide  an  answer  to  the  first 
question.  However,  all  common  sense  seems  to  indicate  that 
the  answer  is  an  emphatic  "yes".  The  model  allows  answers 
to  a  variety  of  questions  which  any  personnel  allocation 
planner  would  need  to  have  answered  relative  to  manning,  man¬ 
power,  and  personnel  subsystem  considerations.  Similarly,  the 
second  question  also  seems  to  be  answerable  in  the  affirmative. 
The  Ship  II  studies  completed  to  date  testify  to  the  ability 
of  Ship  II  to  provide  data  which  are  not  available  from  other 
sources.  This  is  particularly  true  because  the  model  will  be 
most  useful  for  test  of  systems  when  they  are  in  the  concept¬ 
ual  and  design  definition  stages  of  development. 

One  may  also  ask  regarding  the  utility  of  any  data 
set  whether  or  not  the  information  gain  is  worth  Lhe  trouble 
it  takes  to  acquire  the  data.  This  is  essentially  a  cost/ 
benefit  question.  Review  of  the  Ship  II  input  requirements 
suggests  that  a  rather  formidable  task  is  involved  in  de¬ 
veloping  the  input  for  any  specific  simulation.  Accordingly, 
the  benefit  to  the  user  will  have  to  be  great  if  the  cost 
benefit  ratio  is  to  assume  meaningful  values".  (Siegel,  1972, 

P.  9-10) 

F.  Practicality' 


This  section  will  discuss  Ship  II  from  the  viewpoint  of 
the  user  of  Ship  II,  who  must  collect  the  input  data,  run  the 
model  and  interpret  the  output  data.  Regardless  of  how  well 
a  model  satisfies  all  other  evaluation  criteria,  it  must  satis¬ 
fy  certain  time  and  cost  reouirements  if  it  is  going  to  be 
used.  This  section  will  discuss  the  major  factors  which 
affect  the  ease  with  which  Ship  II  can  be  used,  and  the  time 
and  cost  which  are  attendant  to  the  use  of  the  model. 
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1.  Input  Data  Requirements 

The  list  of  the  input  data  requirements  is  shown  in 
Table  19.  In  the  far  left  column,  "C.T."  is  an  abbreviation 
for  card  type,  which  is  the  basic  designator  to  be  used  when 
referring  to  the  different  types  of  input  data.  A  brief  de¬ 
scription  of  each  type  of  input  will  be  given  in  the  following 
pages.  Appendix  A  contains  a  detailed  specification  of  every 
data  element  for  each  card  type. 

a .  Types  of  input  data 

(1)  CT  20  -  Rating  Master  List.  This  data  com¬ 
prises  a  list  of  all  ratings  on  board  the  ship  and  upward 
equivalents,  where  they  exist. 

(2)  CT  21  -  NEC  Master  List.  All  NECs  and  their 
equivalents  are  listed  on  this  card  type. 

(3)  CT  25  -  Ship  Divisions.  The  name  and  number 
of  men  in  each  ship  division  are  required.  In  addition,  this 
CT  includes  scheduling  information  for  divisional  training. 

(4)  CT  26  -  Billet  List.  This  CT  identifies  each 
billet  aboard  the  ship  by  division,  rate,  rating  and  NEC. 
Evolutions  and  personnel  requirements  for  each  are  also  included. 

(5)  CT  27  -  Watch  Station  Manning.  Each  watch 
station  aboard  the  ship  is  specified  and  the  particular  watch- 
standers  for  all  stations  are  specified  for  each  readiness 
condition  for  which  the  station  is  manned. 

(6)  CT  30  -  Subsystem  Data.  The  name  and  number 
of  each  subsystem  on  the  ship  and  the  number  of  equipments 
in  each  is  listed  on  this  CT.  Failure  detection  times  for 
each  steaming  condition  and  readiness  rating  information  is 
also  included  for  each  subsystem. 

(7)  CT  31  -  Equipment  Data.  Each  equipment  is 
first  identified  by  subsystem  number,  equipment  number  and 
name.  The  operation  percentage  of  eauipment  for  each  readi¬ 
ness  condition  is  specified;  this  represents  differential 
utilization  of  equipment  during  the  different  steaming  con¬ 
ditions.  The  parameters  of  the  distribution  of  equipment 
failures  and  deferrals  are  specified  on  this  card  type. 

Each  equipment  can  be  specified  to  be  cither  critical  or 
non-critical . 
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TABLE  19 


SUMMARY  OF  INPUT  DATA  REQUIREMENTS  AND  SOURCES 


C.T. 

DATA  CATEGORY 

20 

Ratinas  Master  List 

21 

NEC  Master  List 

25 

NEC  equivalents 

Ship  division  and  trainina 

26 

information 

Billet  list  (div. ,  man. 

27 

no .  ,  etc. ) 

Augments  and  evolutions 

Watch  station  manning 

30 

Subsystem  data 

31 

Equipment  items 

32 

Watch  station  location 

Operating  %,  MTBF 

Degradation  information 

Deferral  data 

Troubleshooting  data 

& 

33 

Repair/test  data 

34 

PM  data 

35 

Tool  inventory 

40 

FM  &  S/A  task  data 

50 

Training  exercise  reqmts 

60 

Ship  readiness  data 

61 

Subsystem  Readiness  Data 

90 

Scenario  events 

SOURCE 


SMD 

SMD 

NEC  Manual 

SMD 

SMD 

Shipboard  Survey 

SMD 

ROC 

3M 

SMD  or  first-hand 
knowledge 
3M 

CASREPT 

3M 

3M 

NEC  manual,  Quals 
manual,  3M  data, 
first-hand  know¬ 
ledge 
MIPS 

NEC  manual,  Quals 
manual,  first-hand 
knowledge 
SMD 
Space 

Type  commander  instruc' 
tions,  fleet  data 
3M,  CASREPT 
3M ,  CASREPT 

Type  commander  instruc¬ 
tions,  fleet  data 
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(8)  CT  32  -  Troubleshooting  Data.  On  this  card 
type,  rate,  rating  and  NEC  requirements  for  a  troubleshooting 
team  of  up  to  four  men  are  specified.  The  parameters  of  the 
time  in  function  distribution  are  also  specified.  If  desired, 
the  user  can  also  specify  enuipment  and  tool  requirements. 

The  troubleshooting  error  probabilities  are  also  listed  (see 
section  III-B-2) . 

(9)  CT  33  -  Repair/Test  Data.  The  data  in  this 
card  type  are  nearly  the  same  as  required  for  the  trouble¬ 
shooting  function.  The  team  requirements  (rate,  rating, 
and  NEC)  and  tool  and  equipment  requirements  are  specified, 
along  with  parameters  of  the  time  in  function  distribution 
and  the  probabilities  of  repair  errors  (see  section  III-B-2) . 
This  information  is  specified  for  each  equipment. 

(10)  CT  34  -  PM  Data.  Each  individual  PM  job  and 
its  frequency  (daily ,  weekly,  monthly,  quarterly,  semi-annual 
and  annula)  are  first  identified.  The  remaininq  data  is  simi¬ 
lar  to  CT  32  and  33:  personnel  (rates,  rating  and  NEC),  equip¬ 
ment  and  tool  requirements  for  each  PM  job  and  the  parameters 
of  the  time  in  function  distribution,  are  listed. 

(11)  CT  35  -  Tool  Inventory.  This  CT  lists  each 
fcype  of  tool  and  test  ecuipment  and  the  number  of  each  aboard 
the  ship. 


(12 )  CT  40  -  Facilities  Maintenance  and  Support 
and  Administrative  Data.  Each  job  on  this  CT  is  named,  iden- 
tified  as  FM  or  S/A  and  assigned  a  division  responsibility, 

if  appropriate.  The  rate  and  ratinq  for  each  job  is  specified, 
along  with  the  degree  of  upward  absorption  to  be  used.  Fin¬ 
ally,  the  man  hours/day  is  specified  and  the  percent  of  each 
job  which  will  be  carried  over  if  the  job  is  not  completed. 

(13)  CT  50  -  Training  Exercise  Requirements.  Each 
training  exercise  is  listed,  together  with  required  readiness 
condition  and  subsystems. 

(14)  CT  60  -  Ships  Readiness  Data.  This  card  type 
contains  the  rules  to  determine  total  ship  readiness  from  sub¬ 
system  readiness  of  each  mission  related  subsystem. 

(15)  CT  61  -  Subsystem  Readiness  Data,  The  rules 
which  relate  to  availabilitv  of  each  enu.inment  in  a  subsystem 
to  readiness  rating  of  the  subsystem  are  SDecified  on  this  CT. 


7J-; 


(16)  CT  90  -  Scenario.  A  sequential  listing  of 
commands  which  constitute  the  mission  scenario  is  contained 
on  this  CT.  Evolutions,  training  exercises,  sea/port  changes, 
etc.,  are  specified  by  this  CT. 

b.  Summary 

This  list  of  input  data  reauirements  demonstrates 
that  collection  and  preoaration  of  this  input  data  could  con¬ 
stitute  a  major  problem  for  a  Ship  II  user.  The  studies  done 
sc  far  have  shown  that  about  3  man  months  is  usually  required 
to  prepare  a  new  set  of  data  for  a  total  ship  application. 

2 .  Parametric  Variation  Ratios 

The  easiest  way  to  modify  the  model's  input  data  is 
by  varying  the  parametric  variation  ratios  (PVR) .  The  model 
includes  35  PVRs  that  can  be  changed  to  induce  corresponding 
changes  in  the  input  data  categories  upon  which  they  act. 
Making  changes  in  PVRs  iias  the  advantages  of  being  a  fast  and 
simple  way  to  induce  changes  in  whole  categories  of  input  data 
without  tampering  with  the  data  and  without  making  any  changes 
in  modeling  operations.  The  major  drawback  of  this  approach 
is  that  differential  treatment  of  items  within  a  data  cate¬ 
gory  is  not  allowed;  i.e.,  a  change  of  PVR  treats  all  the  data 
in  the  corresponding  data  category  the  same.  If  an  experi¬ 
mental  design  calls  for  some  items  in  a  data  category  to  be 
changed  while  others  are  to  be  changed  in  a  different  way  or 
remain  unchanged,  the  input  data  set  for  the  chosen  category 
must  be  pulled  out  and  new  data  entered  manually  to  satisfy 
the  differential  treatment  called  for  in  the  experimental 
design.  The  35  PVRs  provided  for  in  the  current  model  deter¬ 
mine  the  areas  of  inquiry  that  the  model  can  be  used  to  study 
most  easily,  i.e.,  the  model  is  currently  set  up  to  deal  with 
research  questions  pertaining  to  the  35  PVRs  already  included 
in  the  model.  These  35  PVRs  break  down  into  seven  groupings 
corresponding  to  the  input  card  types  (CT)  they  modify: 
Division  Training  (CT  25) ;  Failure  Detection  (CT  30) ;  Equip¬ 
ment  Data  (CT  31) ;  Troubleshoot  -  CM  (CT  32) ;  Repair/Test  - 
CM  (CT  33);  Preventive  Maintenance  (CT  34);  Facilities  Main¬ 
tenance  and  Support/Administrative  (CT  40) .  Table  20  contains 
a  list  of  input  variables  v/hich  can  be  varied  by  PVRs. 

3 .  Knowledge  of  Computer  Programs. 

Ship  II  is  a  very  large  and  complicated  computer 
program.  It  is  composed  of  128  separate  subroutines  and  a 
total  of  about  24,000  fortran  program  statements.  Like  all 


TABLE  20 


PARAMETRIC  VARIATION  RATIOS 


Training  Session  Duration 
Hours /Week/Man 

Failure  Detection  Times  Under  Cond.  I 
Failure  Detection  Times  Under  Cond.  Ill 
Failure  Detection  Times  Under  Cond.  IV 
Failure  Detection  Times  Under  Cond.  V 
Failure  Detection  Probability  During  PM 
Operation  %  at  Cond.  I 

Operation  %  at  Cond.  Ill 

Operation  %  at  Cond.  IV 

Operation  %  at  Cond.  V 

Mean  Time  Between  Failure 
Probability  of  Degradation 
Deferrals  for  Assistant:  Prob. 

Deferrals  for  Assistant:  Mean 
Deferrals  for  Assistant:  Stan.  Dev. 
Deferrals  for  Parts:  Prob. 

Deferrals  for  Parts:  Mean 
Deferrals  for  Parts:  Stan.  Dev. 

Time  in  Function:  Mean 

Time  in  Function:  Stan.  Dev. 

Error  Probabilitv:  PI 
Error  Probability:  P2 
Error  Probability:  P3 
Error  Probability:  PT 
Delay  Times  .(DT) 

Time  in  Function:  Mean  * 

Time  in  Function:  Stan.  Dev. 

Error  Probability:  PD 
Error  Probability:  P2 
Probability  -  PTR/PTR 
Time  in  Function:  Mean 
Time  in  Function:  Stan.  Dev. 
Manhours/Day 

%  Carryover  for  Ships  Work 


such  programs,  debugging  is  never  finally  completed  in  the 
sense  that  the  model  has  been  run  with  all  ranges  of  values 
of  all  input  variables  and  all  output  variable  values  gene¬ 
rated  under  these  conditions  tested  for  correctness.  As  a 
consequence,  when  a  study  is  done  using  some  model  process 
not  previously  used,  or'  some  extreme  input  values,  the  pos¬ 
sibility  exists  that  a  model  defect  may  be  encountered.  When 
such  a  defect  is  encountered,  the  complexity  of  the  program 
requires  that  debugging  be  done  by  someone  familiar  with  the 
operation  of  Ship  II.  The  probability  of  a  problem  of  this  type 
occurring  is  low  because  Ship  II  has  been  run  a  number  of  times. 
Nevertheless,  it  must  be  considered  as  a  factor  involved  in 
use  of  Ship  II. 

Ship  II  does  have  a  large  number  of  error  messages 
which  will  be  printed  whenever  errors  in  input  data  or  pro¬ 
gram  operation  occur.  However,  manv  of  the  error  messages 
are  defined  in  terms  of  specific  operation  of  particular  sub¬ 
routines  rather  than  in  terms  of  errors  which  the  user  could 
have  made.  For  example,  error  #571  is  described  as  follows: 
"Came  out  of  BUMPABLE  with  a  5";  Error  #590  is:  "Entered  with 
a  man  who  was  not  defined  as  idle  (and  reserved),  or  as  assign¬ 
able  to  special  watch  repair  status."  Error  messages  of  this 
type  convey  very  little  information  to  someone  who  is  not  very 
familiar  with  Ship  II.  They  do  not  make  apparent  whether  the 
error  is  caused  by  some  action  of  the  user  or  by  a  program 
defect. 

4 .  Type  of  Computer 

Ship  II  is  now  programmed  for  running  on  a  CDC  3800 
computer.  The  use  of  Ship  II  on  another  Computer  of  a  different 
word  size  may  require  too  large  a  reprogramming  effort  for  a 
user  who  wants  to  use  the  model  only  for  a  particular  prob¬ 
lem,  but  would  be  worthwhile  for  general  use.  At  present, 
a  potential  user  will  have  to  have  access  to  a  CDC  3800. 

5.  Cost  of  Using  Ship  II 

The  total  cost  of  using  Ship  II  depends  mainly  on*the 
cost  of  preparing  the  input  data,  plus  the  cost  of  computer 
to  exercise  the  model.  The  first  component,  input  data  costs, 
in  some  applications  can  be  the  major  source  of  cost  associated 
with  using  the  model.  Approximately  three  man  months  are 
required  for  someone  faimilar  with  Ship  II  to  prepare  a  com¬ 
plete  data  set;  a  first-time  user  should  perhans  plan  on  three 
and  one-half  to  four  months  for  data  preparation,  unless  he 
is  familiar  with  use  of  Navy  data  systems.  See  section  1  for 
further  discussion  of  input  data  reouirements. 


Computer  costs  for  Ship  II  depend,  of  course,  on  the 
length  of  time  the  program  is  run  on  the  computer.  The  run¬ 
ning  time  for  10  weeks  for  a  total  ship  of  the  size  of  a 
destroyer  is  approximately  7  hours.  A  reasonable  study  for 
Ship  II  could  compare,  say,  three  different  manning  sets  over 
a  50  week  period.  Such  a  study  would  require  105  hours  running 
time.  The  hourly  charge  for  the  CPC  3800  at  Naval  Research 
Laboratory  is  $100.00.  At  this  rate,  the  study  described 
above  would  incur  a  computer  cost  of  $10,500.  A  comnuter 
change  of  $100. 00/hour  is  considerably  lower  than  the  cost 
of  using  a  commercial  CDC  3800.  If  a  user  must  obtain  use  of 
a  computer  by  this  mean.',  the  cost  of  using  Ship  II  could  be 
two  to  three  times  higher. 

a.  Program  Size  Problems 


Ship  II  programs  are  written  in  Fortran  IV  and 
used  on  a  CDC  3800  digital  computer.  That  computer  has  a 
core  storage  of  57,000  storage  locations  (words).  Ship  II, 
on  the  other  hand  requires  approximately  150,000  items  of  in¬ 
put  data.  In  addition,  the  computer  program  itself  is  very 
large  and  requires  another  24,000  computer  core  storage 
locations.  Normally,  each  item  of  data  would  occupy  a  sep¬ 
arate  word  in  computer  core  storage,  as  would  each  program 
statement.  But  in  Ship  II,  the  number  of  data  items  and  pro¬ 
gram  statements  is  far  too  large  for  this  procedure  to  be 
employed.  Two  space  saving  technic. ues  were  used  to  accommo¬ 
date  the  massive  amount  of  data  required:  bit  packing,  and 
the  use  of  overlays.  Bit  packing  refers  to  the  storage  of 
more  than  one  data  item  in  e  single  word  of  core  storage. 
Overlays  are  a  technique  in  which  programs  too  large  to  fit 
in  core  are  divided  into  independent  parts  which  are  called 
and  executed  as  needed.  The  parts  of  the  program  not  stored 
in  core  are  stored  in  slow  access  peripheral  storage,  from 
which  the  programs  are  loaded  into  memory  as  needed. 

Both  bit  racking  and  overlays  increase  the  size 
of  effective  memory  bat  only  at  a  larqe  increase  in  program 
running  time.  When  bit  packing  is  used,  the  input  data  items 
must  be  extracted  from  the  computer  word  in  which  they  ar$ 
stored  every  time  the  data  is  needed.  In  Ship  II,  bit  pack¬ 
ing  and  unpacking  is  done  by  calling  one  of  eight  special 
bit  packing  subroutines.  The  process  of  transferring  control 
to  a  subroutine,  which  involves  transmittal  of  data  from  main 
program  to  the  subroutine,  is  very  slow  compared  with  the 
length  of  time  required  to  process  the  program  statements 
themselves.  Ship  II  has  such  a  large  amount  of  data  that 
extensive  bit  packing  was  required  to  fit  the  computer. 
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Every  subroutine  in  Ship  II  which  uses  input  data  also  uses 
the  bit  packing  subroutines.  Because  these  subroutines  are 
relatively  slow  and  are  used  often,  the  time  to  execute  the 
program  is  considerably  longer  than  the  time  required  just  to 
execute  the  program  statements  themselves. 

The  use  of  overlays  also  increases  the  total  run¬ 
ning  time  of  a  program.  When  a  subroutine  which  is  not  stored 
in  core  storage  is  required,  part  of  the  contents  of  core 
(probably  another  overlay)  are  transferred  out  of  core  to 
peripheral  storage,  and  the  overlay  which  contains  the  needed 
subroutine  is  transferred  into  core  storage.  The  program  then 
proceeds  until  the  next  overlay  is  needed  and  the  process  is 
repeated.  The  problem  with  the  use  of  overlays  is  that  pro¬ 
cess  of  transferring  data  to  and  from  peripherical  storage  is 
much  slower  than  the  time  required  to  execute  individual  pro¬ 
gram  statements. 

The  net  result  of  use  of  bit  packing  and  overlays 
is  that,  while  a  large  amount  of  data  can  be  used  with  a  rela¬ 
tively  small  computer,  the  running  time  for  Ship  II  has  been 
significantly  increased.  Experts  in  Ship  II  programming  have 
estimated  that  over  one-half  of  the  total  running  time  of 
Ship  II  is  contributed  by  bit  packing  and  overlays  alone.  In 
other  v/ords,  if  Ship  II  could  be  run  on  a  computer  with  large 
enough  core  storage  to  eliminate  bit  packing  and  overlays 
(and  such  machines  are  commercially  available)  then  the  run¬ 
ning  time  for  the  model  would  be  reduced  at  least  by  one-half. 

6 .  Output  Data 

a.  Ship  II  Output  Reports 


The  output  data  for  Ship  II  is  presented  in  nine 
separate  reports.  Each  report  is  discussed  in  detail  in 
Appendix  B;  a  brief  description  of  each  is  presented  below. 

(1)  The  Personnel  Record  is  a  summary  of  weekly 
workload  for  each  man  in  nine  different  job  categories. 

• 

( k )  Maintenance  Personnel  Summary  (Rate/RTG)  . s  a 
summary  of  maintenance  personnel  usage  statistics  accumulated 
by  rate  and  rating  of  personnel. 

(3)  Maintenance  Personnel  Summary  (NEC)  is  a  sum¬ 
mary  of  maintenance  Dersonnel  usage  statistics  accumulated 
by  NEC. 


(4)  The  Readiness  Summary  contains  the  Combat 
readiness  history  of  each  equipment  specified  by  the  user  to 
be  a  mission-related  eauipment,  and  for  the  total  ship. 

(5)  The  Equipment  Summary  is  a  report  on  the  avail¬ 
ability,  down  time,  and  components  of  down  time  for  each  item 
of  equipment  simulated. 

(6)  The  Planned  Maintenance  Summary  contains  data 
on  performance  each  planned  maintenance  job.  A  separate  sum¬ 
mary  is  provided  for  daily,  weekly,  monthly,  ouarterly,  semi¬ 
annual,  and  annuel  planned  maintenance  tasks. 

(7)  The  Facilities  Maintenance  and  Support  and 
Administrative  Summary  contains  a  summary  of  performance  of 
each  FM  and  SA  job. 

(8)  The  Divisional  Training  Summary  contains  data 
on  divisional  training  activities. 

(9)  The  Training  Readiness  Summary  reports  on  the 
extent  to  which  training  exercise  demands  were  met. 

b.  Output  Data  Processing 

According  to  Siegel  (1971)  ,  "Ship  II  provides... 
a  rich  variety  of  output  data...."  (P.  5).  A  wide  variety  of 

statistics  needed  by  manpower  planners  is  included  in  its  out¬ 
put.  One  of  the  major  problems  attendent  to  use  of  the  out¬ 
puts  of  Ship  II  is  that  user  has  no  control  over  the  outputs. 
Regardless  of  the  problem  area  being  studied  or  the  interest 
of  the  user,  the  same  output  reports  in  an  invariable  format 
are  always  produced  at  the  end  of  a  run.  'Because  the  output 
of  the  model  cannot  be  selected  by  the  user,  it  was  necessary 
to  insure  that  the  output  would,  contain  information  which 
might  be  required  to  solve  a  wide  variety  of  problems.  As  a 
result,  the  output  is  voluminous.  ror  example,  consider  a 
study  based  on  a  destroyer  with  12  divisions  and  about  100 
people.  This  would  produce  50  pages  of  computer  printout', 
everytime  an  output  command  is  processed.  Although  the  fre¬ 
quency  of  output  can  be  selected  by  the  user,  all  Ship  II* 
studies  to  date  have  printed  a  set  of  outputs  every  seven 
days  of  system  history.  Using  this  output  frequency,  a  study 
covering  three  manning  sets  over  a  50  week  period  would  pro¬ 
duce  a  total  of  7500  pages  of  computer  printout  A  major 
investment  of  time  could  be  required  to  process  this  volume 
of  output  to  obtain  the  data  in  the  form  reauired  by  decision 
makers . 
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G.  Other  Criteria 

This  section  will  briefly  discuss  several  other  criteria 
and  their  implications  for  Ship  II  model  quality.  The  first 
two  criteria  to  be  discussed  are  taken  from  Meister  (1971) . 
Following  these,  the  modiel  evaluation  nuestionnaire  developed 
by  Prather,  (Hutchins,  et.a_l. ,  1973)  will  be  answered  for  Ship  II, 

1 .  System  Development  Applicability 

a.  Comprehens i vene  s  s 

Ship  II  is  a  very  comprehensive  model  within  the 
limits  that  it  is  a  Ship  model.  Any  kind  of  equipment  or  sys¬ 
tem  for  which  the  required  input  data  can  be  specified  can  be 
used  with  Ship  II.  Ship  II  can  be  used  for  ship  types  other 
than  destroyers.  Any  ship  which  can  be  described  by  the  func¬ 
tions  shown  in  Figure  1  can  be  modeled  with  Ship  II.  These 
functions  appear  to  describe  almost  all  Navy  ships  except  the 
small,  special  purpose  ships,  aircraft  carriers,  and  perhaps 
submarines. 

Another  feature  contributing  to  the  comprehensive¬ 
ness  of  Ship  II  is  the  wide  variety  of  shipboard  parameters 
which  can  be  simulated.  The  user  can  specify  any  task  set 
(PM,  FM,  etc.)  task  loading,  manning  set  and  mix,  equipment 
sets,  etc.  that  he  wants,  provided  he  can  provide  the  neces¬ 
sary  input  data  for  these  conditions.  The  basic  limit  on 
manning  variation  is  the  requirement  that  enough  men  must  be 
available  to  stand  all  watches,  but  watch  standing  require¬ 
ments  can  also  be  specified  by^  the  user. 

b.  Applicability 

The  relevance  of  this  criteria  to  Ship  II  is  mar¬ 
ginal.  The  essence  of  this  criteria  is  whether  the  model  pro¬ 
duces  predictions  of  future  system  performance  effectiveness 
or  measures  of  present  system  performance?  Ship  II  falls  in 
the  latter  category:  the  model  outputs  descriptions  of  system 
performance  for  the  period  over  which  the  model  was  run.  • 

c.  Timing 

This  criterion  describes  the  stages  of  system  dur¬ 
ing  which  the  model  can  be  applied.  Ship  II  can  be  applied 
at  any  stage  in  the  system  design  process  for  which  the  re¬ 
quired  manning,  task  and  eauipment  input  data  is  available. 
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This  input  data  is  usually  taken  from  SMD's  and  other  formal 
documents.  However,  Ship  II  can  be  used  before  such  data  is 
available  to  study  tentative  manning  configurations  and  other 
such  problems. 

2.  Model  Characteristics 


a.  Objectivity 

Model  objectivity  refers  to  the  degree  of  subjec¬ 
tive  judgements  which  are  required  to  use  the  model.  Ship  II 
input  data  is  very  flexible  in  terms  of  source  requirements; 
the  data  can  originate  in  a  wide  variety  of  types  of  sources. 
However,  in  the  Ship  II  applications  to  date,  little  of  the 
input  data  has  been  based  on  subjective  judgements.  The 
large  majority  of  data  has  been  obtained  from  3M  tapes, 
CASREPT  system  forms,  SMD's,  and  other  Navy  sources.  Ship  II 
meets  the  objectivity  requirements  for  most  of  the  types  of 
studies  it  was  designed;  however,  in  some  studies,  subjective 
judgements  may  be  required  to  generate  the  input  data  and  run 
the  model. 


b.  Structure 


This  criteria  refers  to  the  extent  to  which  model 
structure  has  been  explicitly  defined  and  described  in  detail. 
The  structure  of  Ship  II  has  been  explicitly  defined  but 
details  of  the  structure  have  not  been  effectively  documented. 
This  fact  was  made  clear  when  a  consultant  for  Ship  II  evalua¬ 
tion  stated  that  it  was  difficult  for  him  to  obtain  enough 
information  to  fully  understand  and  evaluate  Ship  II.  Most 
of  this  information  is,  however,  contained  in  reports  but  is 
difficult  to  extract.  Many  of  the  basic  characteristics  of 
Ship  II  have  been  stated  in  terms  of  changes  to  TSSM,  the 
earlier  version  of  Ship  II  mentioned  in  Chapter  I.  Full 
understanding  of  these  characteristics  requires  knowledge  of 
TSSM,  which  must  be  obtained  from  other,  difficult  to  obtain, 
documents.  In  summary,  Ship  II  model  structure  has  been 
explicitly  defined  but  relevant  information  is  difficult  for 
someone  not  familiar  with  Ship  II  documentation  to  obtain. 

3 .  Evaluation  of  Ship  II  on  Prather  Questionnaire 

This  section  will  list  each  questionnaire  item  and 
the  response  for  Ship  II.  The  ratings  following  each  answer 
indicate  the  certainty  of  the  response;  "A"  means:  "I  am 
positive  of  the  answer",  "B"  means:  "I  am  reasonably  certain, 
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"C"  means 
and  "D" 


"incomplete  data  tends  to  support  the  answer  given", 
means:  "no  information  is  available  on  this  auestion." 


1.  If  the  input  rcauired  so  voluminous  and/or  difficult  to 
obtain  as  to  pose  a  major  factor  to  consider  before  using  the 
model?  Answer:  yes;  rating:  A. 

Comment :  A  list  of  input  data  reauired  r^r  Ship  II  is  given 

earlier  in  this  report.  An  experienced  Ship  II  user  will  usu¬ 
ally  require  approximately  3  man  months  to  prepare  a  complete 
input  data  set  for  Ship  II.  For  most  of  the  input  data  cate¬ 
gories,  data  values  must  be  input  at  the  start  of  a  run;  Ship 
II  will  not  operate  otherwise.  However,  depending  on  the  pur¬ 
pose  of  the  user,  certain  data  values  may  be  arbitrarily  speci¬ 
fied  without  degrading  the  output  from  the  model. 

2.  Are  the  accuracy,  validity,  and  reliability  of  the  input 
data  and  its  source  well  known?  Answer:  yes,  rating:  B. 

Comment:  Input  data  for  Ship  II  are  acauired  from  a  variety 

of  sources  depending  on  the  requirements  of  the  user. 

3.  Does  the  model  presently  receive  data  from  other  computer 
models?  Answer:  no,  rating:  B. 

Comment :  No  data  are  taken  directly  from  other  computer 

models.  But  other  models  could  serve  as  a  data  source  at  the 
user's  discretion. 

4.  Has  a  sound  basis  been  laid  by  the  model  builders  for  any 
non-standard  modeling  techniques  used  or  any  innovative  appli¬ 
cation  of  standard  ones?  Answer :  not  applicable,  rating:  A. 

Comment :  The  simulation  techniques  used  are  straightforward 

and  well  documented. 

5.  Is  the  level  of  sophistication  of  the  technique  too  high 
or  too  low  for  the  needs  of  the  application  area?  Answer: 
yes,  rating:  C. 

Comment:  For  past  applications,  the  model  has  been  unneces¬ 

sarily  sophisticated,  resulting  in  the  user  having  to  specify 
assumptions  to  simplify  the  application.  However ,  it  is  pos¬ 
sible  to  conceive  of  reasonable  problem  classes  which  would 
make  use  of  the  present  level  of  sophistication.  Therefore, 
although  Ship  II  has  been  too  sonhi sticated  for  past  applica¬ 
tions,  future  needs  might  justify  the  level  of  sophistication 
present  in  the  model. 
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6.  Is  the  modeling  technique  used  appropriate  to  the  appli¬ 
cation  area?  Answer:  yes,  rating:  A. 

Comment :  Simulation  is  ideally  suited  to  the  study  of  prob¬ 
lems  involving  many  variables  and  many  complex  interactions 
among  variables. 

7.  Is  the  model  known  to  be  deterministic  or,  if  it  is  stoch¬ 
astic,  is  it  known  to  be  reliable?  Answer:  no,  rating:  C. 

Comment:  A  discussion  of  the  reliability  of  Ship  II  and  a  sum¬ 

marization  of  reliability  data  is  contained  in  Section  III-C. 

8.  Do  the  modeled  system  elements  (e.g.,  blocks  in  the  flow 
diagram)  reflect  what  actually  exists  in  the  Navy?  Answer: 
yes,  rating:  B. 

9.  Do  the  modeled  system  processes  (e.g.,  connecting  lines 
between  blocks  in  the  flow  diagram)  reflect  what  actually  hap¬ 
pens  in  the  Navy?  Answer:  yes,  rating:  B. 

Comment  (8  &  9) :  Many  simplifying  assumptions  are  used  in  Chip 
II.  In  most  cases,  these  assumptions  are  consistent  with  the 
types  of  simplifying  assumptions  usually  required  in  modeling 
complex  systems. 

10.  Is  the  level  of  detail  appropriate  for  the  application? 
Answer:  yes,  rating:  C. 

Comment :  For  many  applications  Ship  II  allows  for  more  detail 

than  is  necessary.  However,  some  detail  can  be  suppressed  by 
the  user  if  he  so  desires. 

11.  Are  all  the  values  assigned  to  such  parameters  as  work 
rates,  fatigue  factors,  storage  capacities,  etc.,  correct? 

Answer:  not  applicable,  rating:  P. 

Comment :  All  such  data  are  specified  and  input  by  the  user, 
and  he  may  change  them,  to  study  their  effects  (sensitivity) . 

12.  If  certain  parameters  are  approximated,  has  an  error  analysis 
ever  been  performed  to  determine  the  cumulative  error  through¬ 
out  the  model  caused  by  these  approximations?  Answer:  No 

Comment:  See  comment  for  item  *11. 


88 


13.  Is  it  known  for  what  parameter  and  .‘nput  values  the  model 
is  valid  and  for  which  values  (possibly  extreme)  it  is  not? 

Answer:  yes,  rating:  C. 

Comment :  A  discussion  of  Ship  II  validity  is  given  in  Section 
B.  However,  previous  studies  seem  to  indicate  that  Ship  II 
is  valid  within  the  range  of  normally  expected  values  of  input 
variables.  But  these  studies  do  not  provide  a  complete  assess¬ 
ment  of  Ship  II  validity. 

14.  Does  the  model  presently  feed  data  to  other  computer  models? 

Answer:  No,  rating:  A. 

Comment :  Outputs  of  Ship  II  are  similar  to  data  which  would 

be  collected  by  a  work  study  team.  They  can  be  utilized  in  a 
variety  of  different  ways  depending  on  the  user.  The  output 
data  could  easily  be  used  to  input  to  other  computer  models 
but  they  are  noc  specifically  designed  for  any  particular 
application. 

15.  Is  the  output  presented  in  such  a  way  that  a  non-computer 
oriented  manager  can,  with  little  or  no  training,  use  it? 

Answer:  yes,  rating:  A. 

Comment :  The  output  data  from  Ship  II  are  formulated  so  as 
to  be  understandable  to  any  Navy  manpower  planner.  Appendix 
B  gives  sample  output  data  sheets  which  will  demonstrate  this 
clarity. 

16.  Are  there  readily  available  results  from  sensitivity  stud¬ 
ies  showing  the  reaction  of  the  models  major  outputs  to  changes 
in  at  least  the  major  parameters  within  reasonable  ranges  of 
values  for  these  parameters?  Answer:  No,  rating:  C. 

Comment :  Some  data  exists  but  it  is  incomplete.  A  full 

discussion  of  this  topic  appears  in  Section  III-D. 

17.  Has  the  model  been  adequately  validated  by  a  comparison 

of  its  results  with  events  in  the  actual  situation  it  attempts  : 

to  model?  Answer:  no,  rating:  B. 

Comment :  A  full  discussion  of  this  topic  appears  in  Section  III-B. 

18.  Has  the  model  ever  been  subjected  to  rigorous  tests  or 
analysis  such  as  might  be  performed  by  a  technical  person  un¬ 
certain  about  or  even  opposed  to  the  model?  Answer:  yes, 
rating:  C. 
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Comment :  During  the  development  of  Ship  II,  Ship  II  was 
extensively  reviewed  by  Navy  personnel.  Several  of  the  per¬ 
sonnel  who  participated  in  this  review  were  opposed  to  the 
use  of  Ship  II  as  a  manpower  planning  aid.  Reviews  of  the 
model  by  these  personnel  were  critical  both  of  model  quality 
and  of  the  use  of  computer  simulation  to  study  Navy  manpower 
problems . 

19.  Were  the  objections  or  auestions  raised  resolved  in  such 
a  way  as  to  give  credibilitv  to  the  model?  Answer:  yes,  rating: 
C. 


Comment :  Some  of  the  critical  comments  raised  during  Navy 

review  were  unjustified,  being  based  on  inaccurate  knowledge 
of  the  uses  of  the  model.  Other  comments  did,  however,  re¬ 
flect  legitimate  flaws  in  model  construction.  The  response 
to  these  comments  was  a  revision  of  the  model.  This  revision 
has  produced  the  la1-"  L  ”ersion  of  the  model,  called  Ship  II. 

In  summary,  then,  of  the  objections  raised  to  the  total  ship 
model  during  Navy  review,  many  have  been  resolved  in  a  way 
supportive  of  the  model.  Legitimate  model  defects  raised  in 
this  review  were  eliminated  by  revision  of  the  model. 

20.  Is  the  model  capable  of  and  worthy  of  being  expanded  or 
otherwise  improved  so  as  to  be  of  greater  benefit  to  the  Navy? 
Answer:  yes.  Rating:  A. 

Comment :  Recent  studies  have  indicated  a  large  potential  for 

improvement  of  Ship  II  which  would  make  the  model  a  more  use¬ 
ful  tool.  Further  comments  on  this  topic  are  contained  in 
Chapter  V. 

21.  In  view  of  the  time,  effort,  and  money  expended  in  use  of 
the  model  and  the  benefits  actually  derived  in  the  application 
area,  is  the  continued  use  of  the  model  warranted?  Answer: 
yes,  rating:  A. 

Comment:  The  types  of  problems  to  which  Ship  II  can  be  applied 

are  discussed  in  Section  E.  Ship  II  can  be  used  to  study  prob¬ 
lems  which  cannot  be  studied  in  other  ways  or  can  be  studied 
only  at  great  expense. 

After  the  auestinnaire  has  been  completed,  the  re¬ 
sponses  are  used  tc  classify  the  model  into  one  of  three 
validity  classes.  The  term  "validity"  is  used  to  mean  over¬ 
all  model  quality.  Class  A  contains  those  models  which  have 
a  solid  technical  foundation  and  have  been  thoroughly  checked 
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TABLE  21 


QUALIFICATIONS  FOR  VALIDITY  CLASS  A 


QUESTION 

2 

4 

5 

6 

7 

8 
9 

10 


ANSWER  AND  RATING 

YES  and  a  or  b 

YES  and  a  or  b  or  not 
applicable 

a  or  b 

YES  and  a  or  b 
YES  and  a  or  b 
YES  and  a  or  b 
YES  and  a  or  b 
a  or  b 


11  YES  and  a  or  b  or  not 

applicable 

12  YES  and  a  or  b  or  not 

applicable 

13  YES  and  a  or  b 

17  YES  and  a  or  b 


18 


19 


YES  and  a  or  b 

(If  question  18  is  answered 
yes  and  a  or  b,  then  ques¬ 
tion  19  must  be  answered  yes 
a  or  b) 

YES  and  a  or  b  or  not*appli- 
cable 


! 


4 
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TABLE  22 


QUALIFICATIONS  FOR  VALIDITY  CLASS  C 


QUESTION 

5 

6 


ANSWER 

Yes  and  a  or  b  and  indication 
that  sophistication  is  too  low 

NO  and  a  or  b 


7  NO  and  a  or  b  and  an  indication 

that  it  is  known  to  be  unreliable 

8  NO  and  a  or  b  and  an  indication  of 

serious  errors 


9  NO  and  a  or  b  and  an  indication  of 

serious  errors 

10  NO  and  a  or  b  and  an  indication 

that  the  level  of  detail  is  too  low 

NO  and  a  or  b  and  an  indication  of 
very  serious  (nearly  irreparable} 
errors 


11 


so  that  the  outputs  are  known  to  be  accurate  and  reliable. 

Class  B  models  are  those  for  which  sufficient  information  about 
the  model  is  not  available  but  major  flaws  have  not  been  shown 
to  exist.  Class  C  models  are  those  which  are  known  to  have 
major  flaws  which  render  them  unsafe  for  operational  use. 

Tables  21  and  22  contain  the  criteria  for  classification  of  a 
model  into  classes  A  and  C.  A  model  which  does  not  fit  either 
class  is  considered  a  Class  B  model. 

The  responses  to  the  questionnaire,  when  scored  accord¬ 
ing  to  these  rules,  show  that  Ship  II  is  a  Class  B  model. 


IV.  CONCLUSIONS 


A.  Validity 

1 .  r’ace  Validity 


The  studies  which  have  been  done  so  far  with  Ship  II 
have  shown  no  major  validity  problems.  However,  these  stud¬ 
ies  have  looked  at  only  a  small  number  of  the  combinations  of 
variables  which  can  be  formed  with  Ship  II.  In  the  seven 
studies  which  were  reviewed,  fifteen  variables  behaved  in  ways 
judged  to  be  contrary  to  expectations;  but  no  consistent  pat¬ 
tern  of  non-validity  is  present  among  the  dependent  variables 
when  compared  across  studies,,  The  number  of  such  relation¬ 
ships  is  small  relative  to  those  judged  to  lend  supoort  to 
model  validity. 

When  interpreting  these  findings,  the  large  number  of 
"inconclusive"  judgements  should  be  considered.  Forty  eight 
dependent-independent  variable  relationships  were  judged  in¬ 
conclusive  -  as  many  as  were  judged  valid.  This  large  number 
is  unsatisfactory  because  it  means  that  the  real  impact  of  the 
Ship  II  studies  on  validity  has  not  been  assessed.  But  it  was 
felt  that  the  results  of  the  validity  survey  would  be  more 
objective  if  only  data  with  a  relatively  clear  cut  interpre¬ 
tation  were  used  to  assess  model  validity.  As  stated  above, 
the  independent-dependent  variable  relationships  in  the  seven 
tables  represent  only  a  small  number  of  the  total  such  combi¬ 
nations  which  could  be  formed  with  the  Ship  II  variables.  There 
fore  no  conclusions  about  Ship  II  as  a  whole  can  be  drawn  from 
these  studies.  The  studies  show  only  that. certain  parts  of  the 
model  may  be  valid,  some  other  parts  may  not  be  valid  and  no 
conclusions  can  be  made  about  validity  of  much  of  the  model. 

2 .  Construct  Validity 

The  construct  validity  of  Ship  II  is  generally  satis¬ 
factory.  All  but  one  of  the  shipboard  functions  in  Ship  II 
adequately  represent  the  performance  of  these  functions  aboard 
the  ship.  The  modeled  functional  performance  is  only  an  approxi 
mation  to  the  real  shipboard  functional  performance  but  such 
approximations  are  an  essential  feature  of  all  simulation 
models.  The  approximations  in  Ship  II  functions  are, in  almost 
vll  cases,  within  the  range  which  allows  the  functions  to  be 
feasibly  modeled  without  distorting  the  output  of  the  function 
to  be  unrepresentative  of  reality. 
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The  one  function  which  needs  improved  construct  validity 
is  the  readiness  scoring  function.  In  Ship  II,  the  readiness 
rating  of  the  ship  now  depends  only  on  equipment  readiness. 

An  unrealistically  simpia  formula  is  used  to  calculate  the  sub¬ 
system  and  total  ship  readiness  rating.  Ship  readiness  on 
actual  ships  depends  on  other  factors  than  equipment  readiness, 
which  are  not  included  in  Ship  II. 


Reliability 


Two  aspects  of  reliability  were  discussed.  One  aspect  of 
model  reliability  mainly  effects  the  cost  of  using  a  model  and, 
conversely,  p  :ision  of  results  obtained  from  a  model.  Little 
data  on  this  aspect  of  Ship  II  is  available  except  that  for  a 
given  number  of  runs,  data  aggregated  over  total  ship  usually 
yields  higher  precision  estimates  than  data  aggregated  over 
the  division  level. 


Another  aspect  of  reliability  is  consistency  of  results 
obtained  by  different  users  on  similar  problems.  This  aspect 
of  Ship  II  was  not  evaluated  because  Ship  II  has  been  used 
only  by  its  developers  and  personnel  from  NAVPERSRANDLAB.  In 
summary,  the  available  data  are  not  sufficient  to  evaluate  Ship 
II  reliability. 


Sensitivity 


The  data  from  most  of  the  Ship  II  studies  were  generally  not 
appropriate  for  evaluating  sensitivity.  Judgments  of  Ship  II 
sensitivity  were  based  on  the  results  of  significance  tests. 

Most  of  the  dependent  variables  studied  were  found  to  be  sen¬ 
sitive  to  the  independent  variables  used,  except  for  the  inde¬ 
pendent  variables,  "mean  time  between  equipment  failures 
(MTBF) " ,  and  "time  in  repair  and  troubleshooting  functions 
(TIF)".  None  of  the  dependent  variables  used  with  these  inde¬ 
pendent  variables  showed  sensitivity.  The  lack  of  sensitivity 
to  these  variables  may  have  been  due  to  .a  masking  variable 
(large  deterals  for  parts)  and  to  small  range  of  variation  in 
MTBF  and  TIF  values-  If  this  is  found  to  be  the  case,  then 
Ship  II  exhibits  adequate  sensitivity  for  most  of  the  vari-* 
ables  studied.  However,  the  input-output  combinations  used 
in  these  studies  comprise  only  a  small  number  of  such  combi¬ 
nations  which  can  be  formed  from  Ship  II;  no  judgments  of 
sensitivity  of  combinations  of  variables  not  used  in  these 
studies  can  be  made. 
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D.  Utility 


An  examination  of  Ship  II  utility  shows  that  Ship  II  < an 
be  applied  to  several  problem  areas  of  current  interest  t 
Navy.  The  four  general  types  of  studies  which  can  be  done 
with  Ship  II  are:  (1)  variation  in  number  and  characteristics 

of  ships  crew,  (2)  variation  in  amount  and  type  of  external 
task  load  imnosed  on  the  crew,  and  personnel  requirements  for 
the  tack  load;  (3)  variation  in  characteristics  of  equipment 
and  personnel  requirements  to  perform  CM  on  the  equipment  and, 
(4)  variation  in  tempo  of  operations,  as  reflected  in  hours 
at  Condition  I,  III  or  IV,  number  of  readiness  exercises,  and 
other  such  variables.  Studies  using  combinations  of  these? 
four  basic  types  can  also  be  used.  All  of  these  studies  can 
be  performed  for  ships  still  being  built,  which  js  a  major 
advantage  of  using  Ship  II. 

Regardless  of  the  type  of  study  dene,  the  following  gen¬ 
eral  types  of  output  variables  indicate  the  operation  of  the 
simulated  ship:  (1)  the  workload  for  each  member  of  the  crew, 
(2)  task  performance  statistics  which  describe  for  each  indi¬ 
vidual  task  of  each  type,  how  many  hours  were  spent  on  the 
task,  how  often  it  was  interrupted,  etc.,  (3)  equipment  state, 
which  describes  the  operational  history  of  each  equipment  in¬ 
cluding  a  summary  of  corrective  maintenance  performance,  and 
(4)  ship  readiness,  including  a  history  of  readiness  rating 
for  total  ship  and  mission  related  equipments  and  a  summary 
of  performance  of  readiness  training  exercises. 

The  rancre  of  problems  which  Ship  II  can  study  and  the 
completeness  of  the  output  data  together  make  Ship  II  a  poten¬ 
tially  useful  tool  of  several  types  of  current  Navy  problems. 

E.  Practicality 

One  of  the  major  factors  which  has  influenced  the  use  of 
Ship  II  is  that  its  practicality  is  low;  Ship  requires  a  large 
amount  cf  data,  part  of  which  is  difficult  to  obtain,  is  costly 
to  run,  and  may  require  extensive  output  data  processing. 
However,  judgements  of  practicality  have  to  be  made  in  terms 
of  other  means  available  to  solve  the  same  types  of  problems. 
For  most  of  the  problems  for  which  Ship  II  can  be  used,  alter¬ 
native  methods  may  be  even  more  costly  and  difficult  to  use 
than  Ship  II.  An  inexpensive,  quick  method  for  determination 
of  the  effect  of  changes  in  crew  composition  of  a  ship  in  terms 
of  the  type  of  output  variables:  used  by  Ship  II  does  not  exist. 
The  major  alternative  to  use  of  Ship  II  would  be  the  use  of 
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work  study  teams.  This  method  can  be  more  costly  than  using 
Ship  II  for  the  same  problem  and  the  data  obtained  may  be  not 
much  better. 

In  summary,  while  the  practicality  of  Ship  II  is  low,  so 
is  the  practicality  of  the  main  alternative  method  for  study¬ 
ing  the  same  problems  Ship  II  studies.  Since  solutions  to 
these  type  of  problems  must  be  obtained,  then  Ship  II  should 
certainly  be  among  the  methods  considered  when  study  of  com¬ 
plex  manpower  problems  is  required. 

F.  Other  Criteria 


1.  System  Development  Applicability 

This  criteria  was  composed  of  three  dimensions;  com¬ 
prehensiveness,  applicability,  and  timing.  Ship  II  is  very 
comprehensive.  Many  different  ship  classes  can  be  studied 
and  the  user  can  specify  an  almost  unlimited  range  of  manning, 
workload  and  equipmen t configurations . 

Relative  to  applicability,  Ship  II  produces  descriptions 
of  current  system  performance;  it  cannot  be  used  to  predict 
future  system  performance.  The  only  limitation  on  the  timing 
in  the  system  design  cycle  when  Ship  II  can  be  applied  is  data 
availability.  Ship  II  has  been  applied  successfully  to  the 
DD-963,  a  ship  which  may  not  be  in  the  water  for  another  two 
years.  The  limit  of  application  in  most  cases  is  that  manning, 
equipment  and  task  input  data  must  be  available. 

2.  Model  Characteristics  ‘ 


This  criteria  is  composed  of  objectivity  and  model  struc 
ture.  Ship  II  satisfies  the  objectivity  criteria  in  that  few 
subjective  judgements  are  required  to  use  the  model  for  most 
problem  areas.  The  structure  of  Ship  II  has  been  explicitly 
defined  but  the  information  describing -model  structure  is 
difficult  to  extract  from  model  documents. 
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V.  DISCUSSION 


A.  Future  Use  of  Ship  II 

The  evidence  presented  in  this  paper  indicates  Ship  II 
has  significant  potential  to  aid  in  solution  of  many  manpower 
planning  problems  for  the  Navy.  Ship  II  can  be  applied  to  a 
variety  of  real  problem  areas  of  interest  to  the  Navy  and  is 
uniquely  suited  for  some  of  these  problems.  For  the  study  of 
ships  in  design  stage,  no  other  techniques  exist  to  obtain  the 
data  which  Ship  II  can  produce,  (Hutchins,  Prather,  Barefoot 
&  Flint,  1973).  With  respect  to  technical  characteristics, 
the  studies  done  to  date  have  revealed  no  major  flaws  in  the 
model;  but  these  studies  have  not  been  sufficient  for  a  com¬ 
plete  evaluation  of  Ship  II.  Ship  II  is  not  by  any  means  a 
perfect  model.  There  probably  are  some  validity  and  sensitivity 
problems  in  the  model.  Collection  of  the  required  input  data 
is  time  consuming.  The  reliability  of  some  input  data  sources 
may  be  low;  but  Ship  II  must  be  evaluated  in  terms  of  auality 
of  outputs  relative  to  other  methods  which  can  be  applied  to 
the  same  problems.  Certainly  Ship  II  produces  information 
which  is  not  completely  valid,  and  which  must  be  interpreted 
carefully  but  so  does  every  other  large  computer  model  and  so 
does  every  other  non-modeling  alternative  to  Ship  II.  If  the 
manpower  problems  of  the  Navy  have  to  be  solved,  which  they  do, 
then  one  should  not  discard  any  potentially  useful  methods  be¬ 
cause  they  are  not  perfect.  Rather  the  best  available  methods 
should  be  used,  even  if  the  best  do  not  have  the  quality  one 
would  desire.  For  certain  types  of  problems,  Ship  II  is  among 
the  best  available  methods  acoessable  to  the  Navy.  The  term 
"best"  is  meant  to  include  both  quality  of  data  and  cost  of 
obtaining  the  data.  Ship  II  can  be  a  useful  tool  which  deserves 
more  use  than  it  has  received  in  the  past. 

Ship  II  can  be  most  effectively  used  to  compare  the  rela¬ 
tive  effects  of  several  input  variable  .configurations.  For 
example,  comparison  of  several  proposed  changes  in  manning 
configurations  or  maintenance  task  assignments  are  problems 
which  are  well  suited  for  Ship  II.  The  model  output  data  fcould 
be  used  to  rank  order  the  changes  with  reasonable  assurance  that 
the  rank  order  represents  reality.  A  user  has  less  assurance 
that  the  absolute  value  of  model  outputs  are  correct.  No 
data  exist,  which  shows  that  Ship  II  can  be  used  to  predict  the 
numerical  values  of  the  dependent  variables  because  Ship  II 
outputs  have  never  been  compared  with  real  shipboard  measure¬ 
ments.  A  user  who  needs  to  predict, for  example,  mean  repair 
times  for  a  given  equipment,  or  total  workload  for  certain 
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ratings  should  regard  the  values  of  these  outputs  produced  by 
Ship  II  as  rough  estimates  only.  The  same  is  true  regarding 
the  numerical  values  of  all  Ship  II  output  variables.  It  may 
be  of  course  that  tie  numeric  values  of  Ship  II  outputs  will 
prove  to  be  good  predictors  of  real  world  data;  but  until  this 
has  been  demonstrated,  the  user  should  treat  the  values  only 
as  rough  estimates. 

1 .  Should  Ship  II  be  Applied  During  System  Development?^ 

Personnel  allocation  and  manning  problems  are  increas¬ 
ingly  under  consideration  during  the  early  phases  of  the  de¬ 
velopment  of  ship  systems.  The  personnel  specialist  is  under 
new  pressures  to  make  quantitative  estimates  of  the  effects 
of  various  manning  and  skill  mixes  on  system  effectiveness. 

What  then  can  be  said  about  the  role  of  Ship  II  for  system 
analytic  purposes  and  about  the  future  use  of  this  model?  The 
mathematically  oriented  sciences,  including  the  related  opera¬ 
tions  research  discipline,  have  made  significant  inroads  into 
effectiveness  prediction  for  the  hardware  aspects  of  modern 
systems.  However,  such  predictions  are  often  found  to  lack 
predictive  precision  when  compared  against  actual  operational 
criteria  of  system  performance.  While  other  factors  certainly 
affect  the  predictive  ability  of  such  techniques,  "human  fac¬ 
tors"  is  one  salient  aspect  which  has  not  often  been  considered 
within  these  techniques.  In  this  context,  "human  factors"  re¬ 
fers  to  the  effects  on  the  operational  effectiveness  of  the 
system  of  such  factors  as  operator  proficiency,  training, 
manning,  and  the  other  variables  and  parameters  of  Ship  II. 

Ship  II,  used  in  conjunction  with  other  digital  simulation 
models  of  man/machine  performance,  can  do  much  to  fill  this 
gap.  Digital  simulation  can  provide  a  tool*  through  which  areas 
of  personnel  failure  can  be  anticipated  and,  hence,  the  human 
contribution  to  system  unreliability  may  be  isolated.  Identi¬ 
fication  of  these  areas,  early  in  the  equipment  development 
cycle,  will  enable  avoidance  of  crew/system  mismatch  and  the 
effects  of  such  mismatching  on  system  reliability. 

Certainly,  such  modeling  will  allow  personnel  planners 
to  do  what  cannot  be  done  without  them.  The  inherent  variability 
of  humans,  their  viability,  and  their  idiosyncratic  tendencies 
make  analytical  or  deterministic  prediction  of  their  behavior 
unrealistic.  Though  their  results  must  be  used  with  care, 
computer  models,  which  allow  for  the  variable  and  often  erratic 
behavior  of  the  human,  seem  more  reasonable  for  the  context  and 
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problem  area  which  is  being  addressed.  As  has  been  pointed 
out  elsewhere,  we  can  pinpoint  a  target  on  the  moon  with  more 
accuracy  than  we  can  prepare  an  economic  forecast  of  the  gross 
national  product.  The  same  may  be  said  about  human  behavior 
in  advanced  systems.  While  manpower  simulation  models  do  not 
represent  a  panacea,  they  are  better  than  nothing  at  all, 
possibly  much  better  than  might  ordinarily  be  anticipated, 
and  certainly  superior  to  "engineering  judgment."  In  the 
areas  of  personnel  and  man/machine  modeling  there  is  clearly 
no  other  mere  suitable  method  of  attack  presently  available. 
After  using  such  models  to  "test"  a  system,  the  areas  receiv¬ 
ing  a  negative  evaluation  may  receive  attention  in  a  number 
of  ways.  Tasks  may  be  reallocated  over  men  and  machines;  or, 
the  design  of  certain  equipment  items  may  be  modified.  Task 
completion  procedures  may  be  modified  so  that  they  will  be 
more  error  free  or  performed  more  quickly.  Sometimes  a  change 
in  the  environmental  conditions,  a  rearrangement  of  the  opera¬ 
tional  equipment,  the  provision  of  alerting  or  warning  devices, 
the  provision  of  job  aids,  modifying  the  level  of  automaticity , 
an  increased  training  emphasis,  or  a  modification  in  the  pro¬ 
posed  manning  may  oe  indiated.  No  matter  what  the  fix  may  be, 
the  effects  of  the  change  may  be  estimated  by  resimulation  and 
estimation  of  whether  or  not  the  modification  has  sufficiently 
increased  system  effectiveness  to  allow  the  meeting  of  design 
objectives. 

The  use  of  Ship  II  in  conjunction  with  other  parallel 
models  is  concordant  with  the  thinking  of  Apostel  (1961) ,  who 
pointed  out  that: 

Modern  technology  utilizes  a  variety  of  models 
in  the  service  of  many  different  needs.  The 
first  requirement  that  a  study  of  model  building 
in  science  should  satisfy  is  not  to  neglect  this 
undeniable  diversity,  and  to  realise  that  the 
same  instrument  cannot  perform  all  these  functions. 

Used  in  conjunction  with  other  specially  developed  simu¬ 
lation  models,  there  is  available  a  relatively  inexpensive  and 
timely  method  for  yielding  broad  insights  into  the  nature  bf 
operator  performance  in  a  man-machine  system.  If  the  real 
system  is  large,  complex,  or  costly,  the  use  of  these  models 
for  a  predevelopment  experiment  may  be  inexpensive  compared 
to  the  trial  and  error  approach  with  the  system  itself;  it  will 
probably  yield  data  for  a  larger  number  of  cases  than  is  prac¬ 
tical  by  other  approaches  and  at  a  lesser  elapsed  time  per 
situation  simulated. 
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The  same  reasoning  applies  to  the  situation  in  which  the 
system  is  in  being  but  is  so  heavily  occupied  that  experimen¬ 
tation  with  changes  in  equipment,  personnel  policy  or  resources 
assignment  rules  may  be  impractical,  expensive,  or  dangerous, 
or  unlawful.  In  these  cases,  too,  available  simulation  tech¬ 
niques  are  appealing. 

As  Shipiro  and  Rogers  (1967)  have  pointed  out,  "...the 
story  of  man's  progress  in  science  and  technology  is  actually 
the  story  of  his  success  in  the  use  of  analogy  and  his  pro¬ 
gress  in  simulation."  Let  us  grant  that  Ship  II,  like  other 
behavioral  models,  is  imperfect.  We  know  the  nature  of  its 
inadequacies.  Accordingly,  it  seems  that  Ship  II  possesses  a 
place  in  the  personnel  subsystem  developmental  scheme.  How¬ 
ever,  if  Ship  II  is  to  be  employed,  it  would  seem  that  such 
modeling  should  be  performed  in  conjunction  with  other  stochastic 
simulation  models  so  that  the  outputs  can  be  cross  checked.  In 
this  way,  the  concurrent  validity  of  Ship  II  will  slowly  become 
established  while  the  tool  is  used  for  the  purpose  for  which  it 
was  established. 

B.  Environment  for  Future  Ship  II  Use 

Ship  II  was  originally  intended  by  the  Navy  to  be  used 
in  an  operational  setting.  The  plan  was  to  develop  a  model 
which  could  be  used  by  manpower  analysts  without  special  train¬ 
ing  in  computer  programming  or  modeling.  The  operation  of  the 
model  would  be  straightforward  and  simple,  with  easily  inter¬ 
pretable  data  being  produced  by  the  model.  Further,  the  model 
was  meant  to  be  a  general  purpose  model,  which  could  be  used  to 
study  many  types  of  shipboard  manning  problems. 

Ship  II  is  useful  for  study  of  several  types  of  manning 
problems,  but  the  model  is  not  appropriate  for  use  as  an  opera¬ 
tional  model,  mainly  because  of  the  complexity  of  the  input 
data  and  the  knowledge  of  the  computer  programs  which  may  be 
required  by  the  user.  It  is  therefore  unrealistic  to  expect 
Ship  II  to  be  used  "off-the-shelf".  A  Ship  II  specialist  will 
have  to  be  involved  in  each  use  of  Ship,  either  as  the  user,  or 
as  a  consultant  to  the  user.  It  is  recommended,  therefore, 
that  Ship  II  continue  to  be  used  at  NAVPERSRANDLAB.  If  if  is 
desired  that  use  of  the  model  be  eventually  transferred  to 
some  other,  more  operationally  oriented  group,  then  a  trainee 
from  that  group  should  be  given  "on-the-job"  training  by  par¬ 
ticipating  in  applications  of  Ship  II  by  NAVPERSRANDLAB.  Even 
if  control  is  eventually  transferred  to  such  a  group,  NAVPERS¬ 
RANDLAB  may  have  to  serve  as  a  consultant  to  the  group  because 
it  may  not  be  feasible  to  instruct  someone  in  the  details  of 
the  model  function  and  program  operation  which  are  sometimes 
required . 
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While  transfer  of  Ship  II  to  an  operational  group  would 
be  feasible  using  the  above  transition  scheme,  the  author  be¬ 
lieves  that  Ship  II  should  not  be  transferred  to  an  operational 
setting  at  all.  Ship  II  was  designed  as  a  research  tool  and  it 
should  be  used  in  an  R&D  environment.  An  operational  model,  by 
contrast,  is  used  in  a  standardized,  routine  way,  using  stand¬ 
ard  inputs  and  outputs.  Ship  II  is  not  meant  to  be  used  in 
this  way.  The  model  is  designed  to  answer  "what-if"  questions 
such  as  what  would  happen  if  manning  were  changed  in  a  certain 
way.  This  type  of  problem  is  a  research  problem;  when  used  to 
study  this  type  of  problem,  Ship  II  is  being  used  as  a  research 
instrument.  Therefore,  Ship  II  can  be  fully  effective  only  when 
used  by  personnel  trained  to  perform  research.  The  skills  re¬ 
quired  of  Ship  II  users  are  similar  to  that  required  by  research¬ 
ers  in  general,  i.e.,  experimental  design,  statistical  analysis 
and  interpretation,  evaluation  of  conflicting  and  uncertain  in¬ 
formation,  and  other  such  skills.  Personnel  with  this  type  of 
training  and  background  are  likely  to  be  found  only  in  R&D 
environments,  such  as  NAVPERSRANDLAB . 

The  statement  that  Ship  II  should  be  used  in  an  R&D  set¬ 
ting  does  not  imply  that  Ship  II  should  be  applied  only  to 
hypothetical  "ivory-tower"  problems.  Indeed,  just  the  opposite 
is  true:  continued  use  of  Ship  II  is  crucially  dependent  on 
establishment  of  much  greater  communication  with  Navy  manpower 
planners  or  project  managers  who  are  faced  with  problems  of 
the  type  to  which  Ship  II  could  be  applied.  Past  use  of  Ship 
II  has  not  been  fully  effective  because  such  communication  has 
not  been  sufficiently  established.  When  NAVPERSRANDLAB  assumed 
responsibility  for  Ship  II,  a  series  of  briefings  were  given  to 
personnel  of  various  Navy  organizations  v/hich  deal  with  manning 
and  personnel.  The  briefings  were  intended  to  inform  these 
organizations  that  Ship  II  was  available  to  assist  them.  The 
response  from  these  briefings  was  apoarently  minimal.  It 
appears  that  very  few  people  outside  of  NAVPERSRANDLAB  and 
Bureau  of  Naval  Personnel  (Pers-A3)  are  aware  that  Ship  II 
exists.  Effective  future  use  of  Ship  II  requires  that  pro¬ 
cedures  be  established  to  bring  together,  in  some  way,  the 
people  with  the  manpower  problems  and  NAVPERSRANDLAB.  It  i,s 
not  sufficient  simply  to  identify  problems;  appropriate  pro¬ 
ject  managers  must  be  identified  and  their  assistance  and 
cooperation  obtained  in  identifying  input  data  sources  and 
collecting  the  required  data.  Lack  of  assistance  by  project 
managers,  or  others  who  have  responsibility  and  knowledge  of 
ship  projects,  to  help  identify  data  sources  and  obtain  the 
data  has  been  a  major  hinderance  to  effective  use  of  Ship  II 
in  past  work. 
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C.  Transfer  of  Ship  II  to  Navy  Personnel  Research  and 

Development  Center 

A  transfer  of  NAVPERSRANDLAB  functions  to  the  Navy  Personnel 
Research  and  Development  Center  will  occur  on  30  June  1973. 

The  question  of  whether  Ship  II  should  also  be  transferred 
rests  partly  on  the  problem  of  changing  the  computer  on  which 
Ship  II  is  used.  Ship  II  is  currently  programmed  for  a  CDC 
3800,  which  has  a  48  bit  word.  If  Ship  II  transfers  to  the 
Center,  the  model  would  be  probably  be  used  on  an  IBM  360,  which 
has  a  32  bit  word.  The  difference  in  word  size  is  crucial  be¬ 
cause  Ship  II  uses  so  much  bit  packing  (see  section  III.  F) . 

The  input  data  is  initially  bit  packed  by  the  input  data  pro¬ 
grams  and  is  then  retrieved  during  program  execution  whenever 
the  data  is  used  by  the  program.  Transfer  of  Ship  II  would  re¬ 
quire  modification  in  the  hit-packing  subroutines  because  of  the 
difference  in  word  size.  V?ithout  these  modifications,  Ship  II 
would  be  unusable  on  an  IBM  360.  Modification  of  Ship  II  to 
run  on  an  IBM  36  0  would  recruire  approximately  6  man  months,  a 
cost  which  does  not  seem  excessive  in  view  of  the  potential 
uses  of  Ship  II.  However,  these  advantages  can  only  be  real¬ 
ized  if  the  users  of  Ship  II  (NAVPERSRANDLAB)  can  become  more 
aware  of  real,  current  Navy  manpower  and  personnel  problems 
to  which  Ship  II  could  be  applied.  The  cost  of  modification 
would  perhaps  be  excessive  is  Ship  II  were  to  be  used  only  to 
study  hypothetical  problems  or  to  collect  data  by  which  to  fur¬ 
ther  evaluate  the  model. 


If  the  problem  of  finding  problems  and  users  for  Ship  II 
can  be  solved,  then  other  factors  weigh  heavily  in  favor  of 
modifying  Ship  II  for  the  IBM  3,60.  The  IBM  360  is  much  larger 
than  the  CDC  3800  currently  being  used;  core  storage  on  the  360 
is  1.5  million  bytes,  which  is  equivalent  to  about  375,000 
words  of  core  storage.  The  CDC  3800  now  being  used  has  57,000 
words.  Because  of  the  larger  core  storage,  Ship  II  as  modified 
for  the  IBM  360  v/ould  use  neither  bit  packing  nor  overlays. 
Elimination  of  the  extensive  use  of  these  techniques  which 
characterize  the  current  version  of  Ship  II  would  reduce  run¬ 
ning  time  by  at  least  a  factor  of  two.  In  other  words  the  modi¬ 
fied  version  of  Ship  II  would  run  at  least  twice  as  fast,  which 
would  reduce  cost  of  using  the  model.  In  addition,  however,  the 
reduced  complexity  of  the  model  would  increase  the  model 
flexibility.  Ship  II  could  be  enlarged  for  use  on  larger  ships 
using  larger  crew  sizes  and  equipment.  Other  modifications  to 
the  model  could  be  made,  if  needed.  The  modified  version  of 
the  model  would  therefore  be  less  expensive  to  operate  and  would 
be  more  flexible  in  the  types  of  problems  it  could  be  applied  to. 


104 


So  far  this  section  has  discussed  the  issues  related  to 
whether  or  not  Ship  II  should  be  reprogrammed .  Another  course 
of  action,  however,  can  be  taken.  Certain  sections  of  the 
model  can  be  initially  modified  and  decisions  on  modification 
of  the  entire  model  can  be  postponed  until  more  information  is 
available.  Ship  II  is  constructed  in  a  modular  fashion;  each 
shipboard  function  in  the  model  is  contained  in  a  separate  sub¬ 
routine.  Each  of  these  subroutines  could  form  the  basis  for 
a  smaller,  more  detailed  model,  which  would  have  two  advan¬ 
tages;  (1)  the  construction  of  new  models  would  be  facilitated, 
and  (2)  if  a  decision  is  made  to  no  longer  use  Ship  II,  the 
model  could  live  on  in  the  form  of  smaller,  more  usable  models. 
The  time  and  effort  spent  to  develop  Ship  II  would  therefore 
not  be  totally  wasted.  This  alternative  seems  to  be  a  reason¬ 
able  course  of  action  in  the  present  situation. 

The  first  priority  for  effective  use  of  Ship  II  is  to  es¬ 
tablish  coordination  between  Ship  II  personnel  and  project 
managers  who  could  sponsor  the  use  of  Ship  II  and  assist  in 
identifying  and  collecting  input  data.  While  this  type  of 
coordination  is  being  established,  selected  sections  of  Ship 
II  could  be  modified  for  IBM  360  and  used  to  form  the  basis 
for  new  models  as  described  above.  Thus  the  construction  of 
new  models  could  be  facilitated. 

An  alternative  course  of  action  may  be  available  for  using 
Ship  II  in  San  Diego.  If  a  CDC  3800  is  available  for  use  in 
San  Diego,  then  Ship  II  could  be  used  there  in  its  present  form. 
Potential  disadvantages  of  this  alternative  are  difficulty  in 
establishing  necessary  coordination,  if  4-he  CDC  3800  is  lo¬ 
cated  in  a  Navy  organization,  or  high  computer  costs,  if  com¬ 
puter  time  is  purchased  commercially.  In  'either  case,  none  of 
the  advantages  of  using  Ship  II  on  a  larger  computer  would  be 
available.  However,  if  Ship  II  will  be  used  only  infrequently 
after  the  new  Center  is  formed,  then  this  may  bo  the  most  pru¬ 
dent  alternative. 

D.  Further  Evaluation  of  Ship  II^ 

The  studies  reviewed  in  Chapter  II  have  provided  a  large 
amount  of  data  about  Ship  II;  but  the  data  have  not  been  suf¬ 
ficient  for  an  in-depth  evaluation  of  the  model  on  all  the 
criteria  discussed  in  Chapter  III.  This  section  will  briefly 


Much  of  this  section  is  based  on  Messer  (1973)  . 
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discuss  work  which  would  contribute  to  further  evaluation  of  Ship 
II.  Given  the  pot  ential  of  Ship  II  to  contribute  signifi¬ 
cantly  to  the  Navy  manpower  planning  process,  the  resources 
needed  to  perform  further  evaluation  appear  to  be  small  in 
comparison  to  the  benefits  which  would  result  from  knowing 
the  validity,  reliability,  and  sensitivity  of  the  Ship  II  inputs 
and  outputs. 

1 .  Predictive  Validity 

As  discussed  earlier  in  this  report,  Ship  II  is  a  com¬ 
plex  model  with  many  input  and  output  variables.  The  valida¬ 
tion  process  would  be  to  run  Ship  II  for  various  combinations 
of  input  and  output  variables  and  to  compare  the  results  with 
actual  "real  world"  data.  The  exhaustive  testing  of  all  input 
and  output  combinations  with  a  model  the  size  of  Ship  II  would 
be  virtually  impossible.  But,  further  steps  can  be  taken  to 
evaluate  the  validity  for  specific  applications  of  Ship  II. 

Although  many  statements  have  been  made  as  to  the  "lack 
of  real  world  data"  and  to  the  !:high  cost  of  obtaining  real 
world  data" ,  adequate  documentation  has  not  yet  been  presented 
that  it  is  impractical  to  obtain  real  world  data  which  could  be 
used  in  evaluating  the  predictive  validity  of  Ship  II.  Siegel 
(1973)  believes  that  "the  difficulty  of  developing  predictive 
validity  data  for  such  models  in  a  Navy  context  does  not  repre¬ 
sent  an  entirely  acceptable  reason  for  failure  to  develop  such 
data.  "  (p.8) 

The  first  task  in  improving  the  evaluation  of  Ship  II 
with  regard  to  predictive  validity  would  be  to  determine  the 
practicality  of  obtaining  real  world  data  to  compare  with  the 
results  of  simulations  using  Ship  II  for  specific  purposes. 

The  "specific  purposes"  would  be  based  on  high  priority  Navy 
manpower  problems.  For  each  specific  purpose,  the  following 
steps  should  be  undertaken: 

a.  Identify  the  types  of  real  world  data  needed  to 
evaluate  the  predictive  validity  of  Shin  II. 

« 

b.  Identify  potential  sources  of  data. 

c.  Identify  deficiencies  and  gaps  in  these  sources. 

d.  Identify  resources  (time,  people,  money)  needed 
to  correct  deficiencies  and/or  fill  gaps. 
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e.  Develop  the  diteria  for  deciding  whether  collect¬ 
ing  real  vvorld  data  is  practical. 

If  it  is  determined  that  it  is  practical  to  collect 
real  world  data,  then  they  should  be  collected  and  compared 
to  the  results  of  the  Ship  II  simulation  and  used  in  evaluat¬ 
ing  the  predictive  validity  of  Ship  II.  Ii  it  is  determined 
that  real  world  data  collection  is  not  practical  (e.g.,  data 
unavailable  and/or  too  costly  to  obtain) ,  then  the  reasons 
why  data  collection  is  not  practical  should  be  carefully  docu¬ 
mented.  In  the  latter  ca^e,  a  less  rigorous  means  of  evalua¬ 
tion  of  predictive  validity  can  be  used,  based  on  what  VanHorn 
(1971)  calls  a  "Turing  test."  VanHorn  describes  a  Turing  test 
as  follows: 

"The  test  is  simple.  Find  people  who  are  directly  involved 
with  the  actual  process.  Ask  them  to  compare  actual  with  simu¬ 
lation  output.  To  make  the  test  a  little  more  rigorous,  one 
might  offer  several  sets  of  simulated  data  and  several  sets  of 
actual  data  and  see  if  the  "experienced"  people  can  tell  which 
is  which.  One  might  even  test  the  classification  for  statisti¬ 
cal  significance.  If  people  can  discriminate,  ask  them  how  they 
do  it.  The  experimenter  can  then  decide  if  the  detectable  dif¬ 
ference  affects  the  inferences  that  he  wishes  to  make....  The 
idea  is  certainly  appealing  and  deserves  further  exploration. 

It  is  probably  a  great  improvement  over  having  the  modeler  use 
his  intuition  to  validate  his  model.  However,  whether  one  can 
make  meaningful  statements  on  the  power  of  such  a  test  is  an 
open  question."  (p.  252-253). 

Another  approach  to  testing  predictive  validity  of  Ship 
II  would  be  to  push  the  model  to  its  limits  by  using  it  for 
unrealistic  purposes  (e.g.,  determine  the  effects  of  cutting 
manpower  on  a  class  of  destroyers  by  three-quarters)  to  deter¬ 
mine  if  the  model  has  predictive  validity  for  wide  ranges  of 
input  variable  values.  In  these  cases  where  extreme  values 
are  used  for  input  variables,  real  world  data  would  not  be 
available  to  use  as  a  comparison  point  for  the  results  obtained 
from  using  Ship  II  simulation.  In  these  cases,  a  judgmental 
evaluation  of  the  predictive  validity  of  Ship  II  should  be 
made  by  comparing  both  the  magnitude  and  the  direction  of  the 
change  in  dependent  variables  due  to  chances  in  independent 
variables  observed  in  the  simulation  to  the  expected  magnitude 
and  direction  of  chances  in  the  real  world. 


2.  Reliability 

Further  evaluation  of  Ship  II  is  needed  with  regard  to 
two  types  of  reliability: 

•  The  reliability  of  the  variances  of  the  output 
variables  in  the  simulation  model  as  compared  to 
the  variances  in  the  real  world. 

•  The  reliability  (or  consistency)  of  the  results  of 
Ship  II  when  it  is  applied  by  different  users. 

With  regard  to  the  reliability  of  the  variances,  the 
same  comments  apply  as  to  the  previous  discussion  of  predictive 
validity.  Although  it  has  been  stated  many  times  that  real 
world  data  on  the  variances  are  not  available  or  that  obtain¬ 
ing  real  world  data  on  the  variances  would  be  "too  costly", 
adequate  documentation  has  not  yet  been  presented  that  it  is 
impractical  to  obtain  real  world  data  on  variances  which  could 
be  used  to  evaluate  the  reliability  of  Ship  II. 

For  such  specific  past  or  future  use  of  Ship  II  simu¬ 
lation,  the  following  steps  should  be  taken  to  attempt  to 
evaluate  the  reliability  of  Ship  II: 

a.  Identify  the  types  of  real  world  variance  data 
needed  to  evaluate  the  reliability  of  Ship  II  (e.g., 
the  variance  of  deferrals  for  parts  and  assistance) . 

b.  Identify  potential  sources  of  data. 

c.  Identify  deficiences  and  gaps  in  these  sources. 

d.  Identify  resources  (time,  people,  money)  needed  to 
correct  deficiencies  and/or  fill  data  gaps. 

e.  Develop  the  criteria  for  deciding  whether  collect¬ 
ing  real  world  variance  data  is  practical. 

» 

If  it  is  determined  that  it  is  practical  to  collect 
real  world  variance  data,  then  they  should  be  collected  and 
compared  to  the  results  of  the  Ship  II  simulation  and  used  in 
evaluating  the  reliability  of  Ship  II.  If  it  is  determined 
that  real  world  data  collection  is  not  practical  (e.g.,  data 
unavailable  and/or  too  costly  to  obtain) ,  then  the  reasons  why 
data  collection  is  not  practical  should  be  carefully  documented. 
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The  reliability  (or  consistency)  of  the  results  of 
Ship  II  when  it  is  applied  by  different  users  has  yet  to  be 
tested.  One  of  the  features  of  Ship  II  is  its  supposed  adapt¬ 
ability  to  other  classes  of  ships  besides  destroyers.  If  fur¬ 
ther  testing  of  Ship  II  is  undertaken,  it  would  be  useful  to: 

•  Estimate  the  extent  of  the  modifications  needed  to 
use  Ship  II  for  other  classes  of  ships. 

•  Estimate  the  resources  (time,  people,  money)  needed 
to  make  the  modifications. 

If  it  is,  in  fact,  feasible  to  utilize  Ship  II  for  other  classes 
of  ships  and  the  model  is  run  for  those  other  classes,  the  reli¬ 
ability  (consistency)  of  the  results  of  Ship  II  when  it  is 
applied  by  different  users  could  be  tested  by  comparing  the 
output  values,  and  the  variances  of  those  output  values,  of 
Ship  II  obtained  from  the  various  classes  of  ships  by  running 
the  model  for  the  same  purpose  (e.g.,  reduced  manning  studies) 
for  those  systems  which  are  the  same  on  both  classes  of  ships. 

3.  Sensitivity 

Further  evaluation  of  Ship  II  is  needed  with  regard  to 
sensitivity.  As  discussed  in  Chapter  III  and  IV  of  this  re¬ 
port,  the  data  available  from  the  previous  Ship  II  studies 
were  usually  not  appropriate  for  evaluating  sensitivity.  Sen¬ 
sitivity  was  defined  as  "the  degree  to  which  the  dependent 
variables  in  a  model  are  responsive  to  the  changes  in  values 
of  the  independent  variables."  Another  way  of  expressing  the 
same  concept  is  to  define  sensitivity  as  the  degree  to  which 
the  results  of  using  the  simulation  model  depend  upon  the  key 
assumptions  made  in  the  simulation  model  logic.  It  is  extremely 
useful  to  know  the  degree  to  which  dependent  variables  are  sen¬ 
sitive  to  changes  in  key  independent  variables.  For  example, 
if  the  results  of  a  simulation  run  for  a  particular  purpose  are 
very  sensitive  to  only  a  few  of  the  key  assumptions  in  the 
model  logic;  i.e.,  the  results  depend  only  on  a  few  of  the 
input  variables,  this  would  raise  the  Question  whether  it  ^ere 
necessary  to  use  as  complex  a  model  as  Ship  II  for  that  particu¬ 
lar  purpose. 

In  future  Ship  II  studies,  the  following  steps  should 
be  taken  to  test  the  sensitivity  of  Ship  II: 
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•  Identify  what  are  assumed  to  be  the  key  assump¬ 
tions  in  the  Ship  II  simulation  model  logic  (these 
would  depend  upon  the  particular  application  of 
Ship  II) . 

•  Run  the  model  for  various  values  of  the  input 
variables  which  comprise  these  key  assumptions. 

•  Indicate  the  degree  to  which  the  predictive  valid¬ 
ity  of  Ship  II  changes  for  particular  applications 
in  response  to  changes  in  the  key  assumptions  in 
the  model  logic;  i.e.,  indicate  the  key  assumptions 
to  which  the  results  of  the  simulation  are  sensitive. 

A.  Utility 

Further  evaluation  of  Ship  II  is  needed  with  regard  to 
utility.  In  Chapter  III  of  this  report,  the  four  general 
classes  of  Ship  II  applications  are  discussed: 

a.  Manning  information 

b.  Equipment  information 

c.  Task  and  training  information 

d.  Operations  requirements  information. 

Although  the  four  classes  of  applications  encompass  a 
range  of  problems  which  could  make  Ship  II  a  potentially  useful 
tool,  an  exhaustive  list  of  the'  potential  users,  their  poten¬ 
tial  uses  of  the  model  to  solve  particular  problems,  and  the 
existence  of  any  alternative  ways  to  solve  these  problems  has 
not  been  developed.  This  information  is  necessary  when  justi¬ 
fying  the  further  development  and  use  of  such  a  highly  complex 
model  which  has  entailed  significant  development  costs  to  date 
and  which  may  entail  further  development-  costs  in  the  future. 

The  following  steps  should  be  undertaken  to  determine 
more  precisely  the  utility  of  Ship  II: 

•  Develop  an  exhaustive  list  of  potential  users  (and 
uses)  of  Ship  II  bv  surveying  relevant  organizations 
in  both  the  sea  and  shore  naval  establishments. 
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Determine  if  the  potential  uses  of  Ship  II  to  solve 
particular  problems  require  a  model  as  sophisti¬ 
cated  as  Ship  II. 

Determine  if  other,  less  complex  methods,  which 
are  valid,  reliable,  and  less  costly,  could  be 
utilized  instead  of  Ship  II. 


5.  Practicality 


* 


I 


f 

f 

I 

1 


*r 

t 


in 

. 


Further  evaluation  of  Ship  II  is  also  needed  with  re¬ 
gard  to  practicality.  In  Chapter  III  of  this  report,  five 
factors  are  responsible  for  the  practicality  of  Ship  II. 

a.  Costly  and  time-consuming  input  data  preparation. 

b.  Complex  computer  programs  which  users  may  need  to 
understand. 

c.  Current  compatability  of  Ship  II  only  with  the  CDC 
3800  computer. 

d.  High  operating  costs. 

e.  Current  inability  of  users  to  select  output  report 
formats. 

In  Chapter  IV,  it  is  stated  that  even  though  the  "practicality 
of  Ship  II  is  low,  so  is  the  practicality  of  most  other  alter¬ 
native  methods  for  studying  the  same  problems  as  Ship  II." 
However,  these  alternative  methods  and  their  degree  of  prac¬ 
ticality  have  not  been  listed  or  discussed  in  detail.  Again, 
this  information  is  necessary  when  -justifying  the  further  de¬ 
velopment  and  use  of  this  complex  and  costly  model. 

The  following  steps  should  be  undertaken  to  determine 
more  precisely  the  practicality  of  Ship- II: 

•  Determine  the  practicality  (cost,  complexity,  and 
so  forth)  of  alternative  methods  to  Ship  II  for 
specific  uses  and  comoare  with  the  practicality  of 
Ship  II. 

•  Based  upon  the  practicality  comparison  and  the 
utility  criteria,  determine  if  further  studies  to 
better  determine  the  validity,  reliability,  and 
sensitivity  of  Ship  II  are  justifiable. 
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E.  Relationship  Between  Ship  II  and  Other  Navy  Models 

Previous  sections  of  the  report  have  argued  that,  even 
though  the  technical  characteristics  of  Ship  II  are  not  com¬ 
pletely  known,  the  utility  of  the  model  is  sufficiently  high 
that  further  study  and  use  of  Ship  II  is  justified.  An  addi¬ 
tional  factor  related  to  this  question  is  whether  the  Navy 
has  other,  better  models  which  can  be  used  instead  of  Ship  II. 

A  comprehensive  survey  of  Navy  manpower  and  personnel  models 
has  just  been  completed  (Hutchins,  Prather,  Barefoot,  Flint, 

&  Letsky,  1973).  One  hundred  and  one  Navy  models,  including 
Ship  II,  were  reviewed  and  categorized  into  one  of  six  cate¬ 
gories.  These  categories  comprise  all  of  the  types  of  man¬ 
power  and  personnel  models  which  are  needed  to  form  an  inte¬ 
grated  and  comprehensive  manpower  planning  system.  Ship  II 
was  classified  as  a  stochastic  Productivity  Measurement  Model. 
Only  eight  other  Navy  models  were  also  classified  as  the  same 
type  of  model.  Only  four  of  these,  including  Ship  II,  are  ship 
models.  Thus  only  three  other  simulation  models  can  potentially 
be  used  in  place  of  Ship  II. 

1.  Description  of  Ship  II  and  Alternative  Models 

Each  of  these  models  will  be  listed  below,  together 
with  a  description  of  model  objective,  primary  inputs  and 
primary  outputs.4  The  same  information-  will  then  be  listed 
for  Ship  II. 

a.  Siegel-Wolf  Two  Man  Operator  Simulation  Model 

(1)  Model  Objective  -  To  predict  via  digital  simu¬ 
lation  the  likelihood  of  successful  man-machine  performance 
and  to  give  equipment  designers  quantitative  answers  to  oper¬ 
ator  overload/underload  conditions  and  the  effect  of  operator 
speed  and  stress  on  system  performance. 

(2)  Primary  Inputs  -  Operator  proficiency;  task 
sequence;  mission  time;  stress  threshold;  parameters. 

- Jk: _ 

2 

The  categories  are  discussed  m  Hutchins,  et  al  (1973)  , 
pp .  56. 

3 

The  other  three  models  are  general  man-machine  models  which 
can  be  applied  to  many  kinds  of  man-machine  systems. 

^taken  from  Hutchins,  et  al  (1973) ,  Appendix  A,  pp.  A475-A482. 
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(3)  Primary  Outputs  -  Percent  success;  tasks 
ignored/failed;  time  spent  on  tasks;  time  spent  waiting; 
average  and  peak  stress. 

b.  Digital  Computer  Simulation  Model  for  Intermediate 

Size  Crews 

(1)  Model  Objective  -  To  predict  via  digital  sim¬ 
ulation  the  operation  or  man-machine  systems  by  crews  of  from 
4  to  20  men. 

(2)  Primary  Inputs  -  Scheduled  events  from  task 
analyst,  documentation,  interviews;  enuipments  as  specified  in 
ship/activity  allowance;  personnel;  emergencies;  parameters. 

(3)  Primary  Outputs  -  Work  unit  detail;  daily 
report;  iteration  report;  run  summaries. 

c.  Large  Group  Oriented  Crew  Behavioral  Simulation 

Model 

(1)  Model  Objective  -  To  predict  via  digital  sim¬ 
ulation  the  man-machine  system  efficiency  and  related  equip¬ 
ment  and  performance  measures  as  a  function  of  crew  composition, 
working  levels  and  stress  imposed  by  the  mission. 

(2)  Primary  Inputs  -  Equipment;  personnel  data; 
parameters;  constants  and  mission  data. 

(3)  Primary  Outputs  -  Work  out  detail;  daily  report; 
iteration  summary  and  run  summary. 

d.  Ship  II 

(1)  Model  Objectives  -  To  simulate  manpower 
related  variables  for  total  ships  for  research  in  manpower 
questions. 

(2)  Primary  Inputs  -  Manning  (billets,  organisa¬ 
tion)  ,  watches  (acquired  from  SMD  or  proposed  manning;  Equip¬ 
ment  data  (MTBF,  CM,  PM)  (acauired  from  SMD,  work  study); 
Operational  scenario  (acquired  from  deck  logs,  Type  Comman¬ 
ders  Instructions) . 

(3)  Primary  Outputs  -  Manpower  utilization  (man¬ 
hours  by  each  man  in  task  areas);  enuipment  data  (failures, 
downtime);  task  data  (tasks  dovie,  postponed,  or  canceled); 
readiness  (equipment  and  total  ship) . 
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Comparison  Between  Ship  II  and  Alternative  Models 


The  information  in  the  previous  paragraphs  shows  that 
none  of  the  three  alternative  models  is  designed  to  perform 
the  same  functions  which  Ship  II  performs.  These  models  are 
useful  and  well  constructed,  but  they  simply  cannot  study  the 
range  of  problems  which  Ship  II  can  study  nor  can  they  provide 
t’<?.  rich  variety  of  output  data  which  Ship  II  provides.  The 
'y  has  no  other  models  which  can  be  used  in  place  of  Ship  II 
:c  most  of  the  problem  areas  Ship  II  was  designed  to  study. 

II  is,  in  fact,  unique. 


VI .  RECOMMENDATIONS 


1.  Ship  II  is  a  potentially  useful  manpower  planning  tool 
which  has  capabilities  not  available  in  any  other  Navy  model. 

An  effort  should  be  made  to  inform  the  Navy  manpower  planning 
community  about  Ship  II  characteristics,  uses,  and  how  the 
mod^l  interfaces  with  other  manpower  planning  models,  such  as 
the  Manpower  Determination  Model  (MDM) .  Special  care  should 
be  taken  to  inform  potential  users  that  Ship  II  is  not,  and 
was  not  designed  to  be,  a  replacement  for  MDM  or  other  such 
models.  Instead  of  competing  with  present  manpower  study 
methods.  Ship  II  provides  an  additional  capability  which  could 
provide  valuable  support  to  Navy  manpower  planners. 

2.  Additional  studies  to  further  evaluate  Ship  II  should  be 
done.  Validity  should  be  of  primary  importance  in  these  stud¬ 
ies,  followed  by  sensitivity  and  reliability.  Additional  data 
on  utility  of  Ship  II  tor  specific  problems  of  current  interest 
to  the  Navy,  and  practicality  of  Ship  II  relative  to  other 
competing  methods  would  also  be  valuable.  The  validity,  sen¬ 
sitivity  and  reliability  studies  should  be  conducted  within 
the  context  of  specific  problem  areas  identified  in  the  utility 
studies.  Performance  of  all  evaluation  studies  should  depend 
on  indications  of  acceptance  of  Ship  II  by  Navy  manpower 
planners. 

3.  Ship  II  should  be  transferred  to  the  new  Navy  Personnel 
Research  and  Development  Center  and  modified  to  run  on  the 
Center  computer.  A  feasible  alternative  to  this  recommenda¬ 
tion  may  be  to  keep  Ship  II  ih  its  present  form  if  the  use  of 
a  CDC  3800  can  be  obtained  in  the  San  Diego  area  at  reasonable 
cost.  If  neither  of  these  actions  is  taken,  the  result  would 
be  that  San  Diego  would  have  the  simulation  function  and  the 
personnel,  but  no  computei  facility;  Washington,  on  the  other 
hand,  would  have  an  appropriate  computer  facility  but  no  per¬ 
sonnel  or  simulation  functions.  The  result  might  be  that  Ship 
II  would  simply  die  of  disuse. 
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APPENDIX  A 


A.  Introduction 

This  section  contains  a  listing  and  description  of  each  input  data 
element.  Input  data  to  the  model  are  entered  via  special  forms  speci¬ 
fically  designed  for  the  SHIP  II  model.  Seventeen  different  forms  are 
used  and,  with  the  exception  of  GENERAL  DATA,  each  represents  a 
different  card  type.  Fifteen  card  types  are  used  for  GENERAL  DATA. 


The  different  forms  are: 

FORM  CARD  TYPE 


GENERAL  DATA  00  —  14 

RATING  MASTER  LIST  20 

NEC  MASTER  LIST 

AND  EQUIVALENTS  21 

SHIP  DIVISIONS  AND 

DIVISIONAL  TRAINING  25 

PERSONNEL  LIST  26 

WATCH  STATION  MANNING  27 

SUBSYSTEM  IDENTIFICATION  30 

EQUIPMENT  DATA  31 

TROUBLESHOOT  (C  M)  32 

REPAIR/TEST  (CM)  33 

PLANNED  MAINTENANCE  (PM)  34 

TOOLS  INVENTORY  35 

FACILITIES  MAINTENANCE  (FM) 

AND  SUPPORT  AND  MAINTENANCE  40 

TRAINING  EXERCISE  REQUIREMENTS  50 

TOTAL  SHIP  READINESS  SCORING  60 

MISSION  FUNCTION  READINESS 

SCORING  61 

EMPLOYMENT  SCHEDULE  90 


In  the  following,  each  of  these  forms  is  presented  along  with  a 
description  of  the  individual  entries.  In  each  case,  sample  data  entries 
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j  iu-'  »L*m  !npjn  wpp ww 


V  'Ji  u' 


FTWF71 


WTW 


have  been  made  to  aid  the  user  in  understanding  the  format  of  each 
element.  The  sample  data  shown  were  used  to  check  out  the  model, 
and  correspond  to  the  sample  output  data  shown  in  sectionTT 

FORMAT  CONVENTIONS 


The  following  conventions  are  applicable  to  all  data  entries. 

1.  All  times  are  entered  in  hours,  unless  otherwise 
noted. 

2.  All  ratings  are  left  adjusted,  and  all  rates  are 
right  adjusted. 

3.  All  numeric  fields  are  right  adjusted. 

4.  Blanks  are  interpreted  as  zeros. 
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B.  Input  Forms 

1.  GENERAL  DATA 

CARD  TYPES,  0  through  14,  GENERAL  DATA,  contain  a  series 

of  constants  related  to  table  sizes,  parameter  variations  and  work  day 

constraints.  The  individual  elements  are  discussed  below. 

NO.  OF  DIFF.  RATINGS  The  number  of  different  skill  types  (ratings) 

aboard.  This  number  should  correspond  to 
the  number  of  ratings  in  the  Rating  Master 
List  (Card  Type  20). 

NO.  OF  DIFF.  NECS  The  number  of  different  Naval  Enlisted 

Classifications  (NEC)  aboard.  Corresponds 
to  the  NEC  Master  List  (Card  Type  21). 

NO.  OF  DIVISIONS  The  number  of  different  divisional  organi¬ 

zations  aboard  (Card  Type  25). 

NO.  OF  WATCII  STATIONS  The  total  number  of  watch  stations  at 

General  Quarters  ( Card  Type  27).  . 

NO.  OF  SUBSYSTEMS  The  total  number  of  subsystems  represented 

by  the  ship's  equipment.  This  value  is  used 
to  read  subsystem  related  data. (Card  Type  30). 

NO.  OF  TOOL  TYPES  The  number  of  different  special  tools  and 

test  equipment  used  in  the  performance  of 
maintenance  (Card  Type  35). 

NO.  PF  FM-S/A  JOBS  The  total  number  of  different  Facilities 

Maintenance  and  Support /Administrative 
jobs  (Card  Type  40). 

The  number  of  different  types  of  exercises 
which  can  be  scheduled  (  Card  Types  50 
and  90). 

A. 3 


NO.  OF  TRAINING 
EXERCISE  TYPES 


NO.  OF  TOTAL  SHIP  The  number  of  decisions  used  to  score  the 

SCORING  DECISIONS  readiness  of  the  total  ship  (  Card  Type  60). 

RANDOM  NUMBER  SEED  A  number  which  initiates  the  sequence  of 

random  numbers.  A  different  number  will 
result  in  a  different  series. 

NO.  OF  MISSION  The  number  of  subsystems  which  are  desig- 

FUNCTIONS  nated  as  mission  functions. 

TOTAL  NUMBER  OF  MEN  The  number  of  men  comprising  the  crew. 

/ 

SUBSYSTEMS  WHICH  The  subsystem  numbers  which  are  designated 

ARE  MISSION  FUNCTIONS  as  mission  functions. 

PLANNED  MAINTENANCE  The  number  of  each  type  of  Planned  Mainte- 

NO.  nance  jobs  (,  Card  Type  34). 

CANCEL  AFTER  The  length  of  time  a  Planned  Maintenance 

job  can  be  delayed  before  it  is  cancelled. 

STANDARD  NAVY  WORK 

WEEK 

WATCHSTANDERS  The  standard  Navy  Work  week  for  Watch- 

standers  at-sea. 

NON-WATCHSTANDERS  The  standard  Navy  Work  week  for  non- 

watchstanders  at-sea. 

PARAMETER  VARIATION  Factors  which  can  be  used  to  alter  variables 

RATIOS  on  the  model  parameters.  If  set  to  1.0  no 

change  will  occur;  0.  8  represents  a  20% 
reduction;  and  1.  2  represents  a  20%  increase. 
The  model  parameters  which  correspond  to 
the  PVR’s  are  listed  in  Table  II. 
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PVR 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 
11 
12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 

34 

35 

36 

37 

38 


TABLE  II 


Card 

JLmg— 

25 

25 

30 

30 

30 

30 

30 

31 
31 
31 
31 
31 
31 
31 
31 
31 
31 
31 

31 

32 
32 
32 
32 
32 
32 

32 

33 
33 
33 
33 

33 

34 
34 
40 
40 

Unused 

Unused 

Unused 


Parameter  Variation  Ratios 

_ Variable  Ident.  _ 

Session  Duration 
Hours /Week/ Man 
Fail  Detect.  Time  Cond.I 


It 

If 

"  ”  III 

tl 

II 

"  V  rv 

II 

ft 

"  »  v 

ft 

It 

Prob.  During  PM 

Op. 

%  at  Cond.  I 

II 

It  ft 

"  III 

II 

II  It 

"  IV 

It 

ft  tf 

"  V 

MTBF 

Prob.  Degradation 
Defer  for  Assist:  Prob. 

"  "  "  Mean 

"  "  "  Stan.  Dev. 

Defer  for  Parts:  Prob. 

"  "  "  Mean 

"  "  "  Stan.  Dev. 

Time  in  Func. :  Mean 
"  "  "  Stan.  Dev. 


Error  Prob: 

PI 

it  ii 

P2 

•i  ti 

P3 

ii  ti 

PT 

Delay  Time  (DT) 

Time  in  Func. :  Mean 
"  "  "  Stan.  Dev. 

Error  Prob. :  PD 
"  "  P2 

Probability  -  PTR/PTR 
Time  in  Func:  Mean 
"  "  "  Stan.  Dev. 

Manhours  /  day 
%  Carry 


Data  Maximum 
40. 95  hours 
4  0.  95  hours 
10. 23  hours 
10. 23  hours 
10. 23  hours 
10. 23  hours 
100% 

100% 

100% 

100% 

100% 

10, 485. 75  hours 
100% 

100% 

10, 485. 75  hours 
10,  485. 75  hours 
100% 

10,  485. 75  hours 
10, 485. 75  hours 
167,772. 15  hours 
167,772. 15  hours 
100% 

100% 

100% 

100% 

655. 35  hours 
167,772.  15  hours 
167,772. 15  hours 
100% 

100% 

100% 

167,772.15  hours 
167,772. 15  hours 
167,772.15  hours 
100% 


JOB  ASSIGNMENT  WORK  Times  which  represent  the  beginning  and 
DAYS  end  of  the  work  day  for  each  of  the  job  types 

listed.  Assignments  will  not  be  made  before 
the  values  labled  "DON'T  ASSIGN  BEFORE" 
or  after  "DON'T  ASSIGN  AFTER".  The  job 
types  are: 


FM 

S/A 

DIV  TRAIN 
PM- PORT 
NOT  DAILY 
DAILY 

PM  SEA/ PORT 

CM 

NON  CRIT 


Facilities  Maintenance 
Support/Administrative  work 
Divisional  Training 

PM  jobs  designated  as  performed  in-port. 
PM  jobs  with  frequencies  other  than  Daily. 
Daily  PM  jobs 

PM  jobs  designated  as  performed  at-sea 
or  in- port. 

Corrective  Maintenance 
Non-critical  corrective  maintenance 

Critical  corrective  maintenance  jobs 


CRIT 


P  SIMULATION  MODEL  INPUT 

PERSONNEL  RESEARCH  AND  DEVELOPMENT  LABORATORY 


JOB  ASSIGNMENT  WORK  DAYS 

OIV  I  PM -PORT  I  PM-SEA/PORT 


2.  RATING  MASTER  LIST 


CARD  TYPE  20  contains  a  list  of  each  rating  represented  in  the 
Watch  Bill,  or  expected  to  be  on  board.  Where  a  rating  has  an  equivalent 
which  would  be  used  in  reassigning  jobs,  that  rating  should  be  listed 
in  the  equivalent  rating  column.  Primary  ratings  should  be  listed  in 
alphabetical  order.  Up  to  75  different  ratings  can  be  entered*  it  is 
advisable  to  list  all  possible  ratings  since  this  information  is  used  to 
screen  other  input  data  for  erroneous  entries.  For  example,  if  in  the 
Personnel  List  a  rating  is  called  for  which  is  not  entered  on  card  type  20, 
an  error  will  result. 

SEQUENCED  RATING  A  list  of  all  ratings  to  be  found  on 

(75)  board.  Limit  =  75. 
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SHIP  SIMULATION  MODEL  INPUT 

NAVAL  PERSONNEL  RESEARCH  AND  DEVELOPMENT  LABORATORY 


3.  NEC  MASTER  LIST  AND  EQUIVALENTS 


CARD  TYPE  21  is  a  sequential  listing  of  the  NEC's  (Naval  Enlisted 
Classification)  whicn  are  held  by  memebers  of  the  crew.  Also  listed 
is  an  equivalent  NEC,  if  one  exists.  Equivalent  NEC's  are  generally 
held  by  higher  rates  whose  NEC's  absorb  lower  NEC's.  The  equivalence 
table  is  provided  for  job  reassignment  in  the  event  the  specified  NEC 
is  occupied. 

SEQUENCED  NEC's  A  list  of  all  NEC's  aboard  ship. 

(75) 

EQUIVALENT  NEC  An  NEC  which  is  equivalent  to  the  one 

(75)  identified  in  the  list  of  sequenced  NEC's. 
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4.  SHIP.  DIVISIONS  AND  DIVISIONAL  TRAINING 
CARD  TYPE  25, 

is  a  sequential  listing  of  divisions  which  provides  division  name,  the 
number  of  personnel  in  each  division,  and  divisional  training  data. 
Divisional  training  data  include  session  duration,  rescheduling  intervals 
minimum  class  size  and  hours  per  week  per  man  of  required  divisional 
training. 

DIV  NO  The  identification  number  of  the  division. 

(20)  This  number  should  be  identical  to  other 

division  numbers  entered  elsewhere  in 
the  input  data. 

DIVISION  NAME  A  sixteen  character  field  which  contains 

the  name  of  the  division.  This  name  is 
used  for  output  identification  only. 

NO.  OF  MEN  The  number  of  men  in  the  division. 

SESSION  DURATION  The  length  of  a  divisional  training  session. 

(HRS. )  This  figure  is  used  as  a  constant  and  all 

sessions  are  of  this  length  (not  randomized). 

If  an  occasion  arises  when  divisional 
training  cannot  be  held,  the  activity  is 
rescheduled.  This  value  represents  the 
amount  of  time  to  wait  before  attempting 
a  reschedule. 

The  minimum  number  of  men  which  can 
constitute  a  training  session.  Unless  this 
number  of  men  are  available  the  session 
will  be  delayed. 

HOURS/WEEK/MAN  Desired  training  for  each  man  each  week. 


MIN.  MEN  PER 
SESSION 


INTERVAL  BETWEEN 
ATTEMPTS  TO 
SCHEDULE  (HRS. ) 


SHIP  DIVISIONS  AND  DIVISIONAL  TRAINING 


5.  BILLET  IDENTIFICATION  AND  AUG.  /EVOL.  STATION  ASSIGNMENT 

CARD  TYPE  26,  lists,  for  each  man  aboard  ship,  his  billet 
title,  skill  and  NEC  identification  and  whether  or  not  he  is  assigned  to 
a  station  for  watch  augmentations  or  evolutions.  This  card  differs 
from  card  type  27;  on  card  type  27  the  title  is  associated-  with 
a  station,  not  a  billet  title. 

DIV  NO  Division  number  (1  through  "N") 

(20) 

MAN  NO  The  sequential  number  of  the  man  in  the 

(350)  division  (same  as  card  type  27).  Nuraoer- 

ing  starts  over  with  each  division. 

BILLET  TITLE  The  title  of  the  billet.  Any  designation 

may  be  used. 

SKILL- RATING  The  rating  of  the  man,  if  rated.  If  he  is 

not  rated,  leave  blank. 

SKILL- RATE  Skill  level  of  the  man. 

NEC  (PRIMARY)  The  man's  primary  Naval  Enlisted 

Classification  (NEC). 

NEC  (SECONDARY)  The  man's  secondary  NEC,  if  he  has  one. 

AUGMENTS  Watches  can  be  augmented  for  training 

exercises.  Up  to  five  different  augmen¬ 
tations  are  possible.  A  "1"  entered  in 
one  of  these  columns  means  that  the  man 
will  be  placed  on  Watch  (Training  Exer¬ 
cise)  if  that  augmentation  is  called  for  by 
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a  Training  Exercise.  Examples  of  aug 
mentation  are  I-ASW  and  I-AAW. 


EVOLUTIONS 


Up  to  20  different  types  of  evolutions  are 
possible.  Evolutions  are  scheduled  as 
part  of  the  employment  schedule.  As  with 
Watch  Augmentations,  a  "1"  in  any  column 
means  that  that  man  will  be  assigned  to  a 
station  for  that  evolution.  Simultaneous 
evolutions  are  not  allowed  except  by  com¬ 
bining  the  stations  and  identifying  the 
combination  as  one  evolution.  Training 
Exercises  and  Evolutions  cannot  occur 
simultaneously. 
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PERSONNEL  LIST 


6.  WATCH  STATION  MANNING 

CARD  TYPE  27  identifies  the 

manning  of  watch  stations  for  each  of  the  primary  readiness  conditions. 
Each  watch  station  on  the  ship  is  listed  and  identified  with  a  title.  The 
manning  of  that  station  is  identified  for  conditions  I,  III,  IV,  and  V. 

If  the  station  is  not  manned  at  one  or  more  of  these  conditions,  the  field 
is  left  blank.  The  personnel  who  man  the  station  are  identified  by 
division  number  and  man  number  within  that  division.  Watch  sections 
are  also  identified. 

STATION  NUMBER  A  station  is  a  position  on  the  ship  which  is 

(350)  manned  around  the  clock  in  at  least  one  of 

the  four  primary  readiness  conditions. 

All  such  stations  are  listed  and  numbered 
in  a  sequential  manner.  This  station 
number  is  used  as  an  index  for  assigning 
personnel. 

STATION  TITLE  A  sixteen  character  field  which  contains 

the  title  of  the  station.  Any  appropriate 
title  may  be  used. 

READINESS  CONDITION  If  a  station  is  manned  during  one  of  the 
MANNING  ASSIGNMENTS  four  readiness  conditions,  the  assigned 

man  is  designated  by  his  division  and 
man  number  (card  type  26).  A.  division 
and  man  number  is  given  for  each  section, 
or  shift.  Condition  I  has  only  one  section 
and  so  relief  is  not  required.  Conditions 
III  and  IV  have  three  sections  and  condition 
V  has  four.  The  sections  in  condition  V 
are  known  as  duty  sections. 
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7.  SUBSYSTEM  IDENTIFICATION 


CARD  TYPE  30. contains  data  on 

the  name  of  each  subsystem  as  well  as  failure  detection  times  and  prob¬ 
abilities,  the  number  of  equipment  items  it  includes,  and  the  number  of 
readiness  decisions  entered  on  card  type  61.  A  subsystem  may  or  may 
not  be  a  mission-oriented  function.  The  subsystem  numbers  of  the 
mission  functions  are  defined  on  the  General  Data  Card.  If  a  subsystem 
is  not  a  mission- oriented  function,  the  entries  related  to  readiness 
decisions  are  not  required. 

SUB  SYS  NO  The  sequential  number  of  the  subsystem. 

(100) 

SUBSYSTEM  NAME  A  sixteen  character  field  which  contains 

the  name  of  the  subsystem.  Any  appro¬ 
priate  name  may  be  entered. 

The  expected  time  in  hours  between  the 
occurrence  of  a  malfunction  and  the  detec¬ 
tion  of  that  malfunction  by  the  crew. 

Four  values  may  be  entered,  one  for  each 
of  the  four  basic  readiness  conditions. 

FAILURE  DETECTION-  The  probability  that  an  undetected  failure 
PROB.  DURING  PM  in  a  subsystem  will  be  detected  during  a 

planned  maintenance  action. 

NO.  EQUIP.  ITEMS  The  number  of  equipment  items  within 

the  subsystem. 

The  number  of  readiness  decisions  for 
each  of  the  four  readiness  states  (Cl,  C2, 
C3,  C4)  entered  on  card  tyr>e  61  for 
mission-oriented  subsystems. 


FAILURE  DETECTION 
TIME 


NO.  READINESS 
RATING  DECISIONS 


FAILURE  DETECTION 


8.  EQUIPMENT  DATA 

CARD  TYPE  31  contains  detailed  reliability, 
failure  effects,  and  logistics  delay  data  for  each  item  of  equipment.  Up 
to  500  items  of  equipment  can  be  represented.  In  addition,  a  special 
data  element  is  provided  which  designates  which  watchstanders  do 
maintenance  jobs,  if  called  for. 

SUBYS.  NO.  A  sequential  number  identifying  the  sub- 

(100)  system  (same  as  entered  in  card  type  30). 

EQUIP.  NO.  The  equipment  number  within  the  subsystem. 

(Total  must  not  Numbering  starts  over  with  each  subsystem, 

be  more  than  The  last  equipment  number  must  correspond 

500)  to  the  number  of  equipment  items  per  sub¬ 

system  entered  on  card  type  30. 

EQUIPMENT  NAME  A  sixteen  character  field  which  contains 

the  subsystem  name.  Any  appropriate 
name  may  be  used. 

In  some  instances,  a  failure  may  occur  in 
a  system  and  the  only  qualified  technician 
is  standing  watch.  If  he  uses  that  equip¬ 
ment  to  perform  his  watch  duties,  the 
model  is  designed  to  assign  him  to  its 
repair.  If  this  policy  is  to  be  used,  this 
value  represents  the  watch  station  location, 
as  identified  on  card  type  26,  which  utilizes 
the  equipment. 

OPERATING  %  OF  The  percent  of  time  the  equipment  operates 

EQUIPMENT  AT  during  each  of  the  four  basic  readiness 

READINESS  CONDITIONS  conditions. 


WATCH  STAT  LOC. 
(IF  APPL) 
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MTBF  (HRS.  ) 

PROB.  DEGR. 

CRITICAL 

DEFERRALS  FOR: 
ASSITANCE  PROB. 

MEAN,  STD.  DEV. 

PARTS  -  PROB, 
MEAN,  STD.  DEV. 


The  Mean  Time  Between  Failure,  based  on 
operating  hours,  of  the  equipment. 

The  probability  that  a  failure  in  the  equip¬ 
ment  would  result  in  a  degradation  of  the 
readiness  of  the  associated  mission  function 
(subsystem). 

An  indicator,  0  or  1,  which  designates  the 
equipment  as  critical  or  non-critical. 
Critical  equipment  items  are  maintained 
any  time  a  failure  occurs  and  take  priority 
over  other  jobs. 


The  probability  that  repair  of  this  equip¬ 
ment  item  will  be  deferred  for  outside 
assistance. 

The  mean  and  standard  deviation  of  the 
time  spent  in  a  deferral  for  assistance 
(hours). 

Same  as  above,  except  the  cause  of  deferral 
is  lack  of  parts. 
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9.  TROUBLESHOOT  (CM) 


CARD  TYPE  32  contains  the  data 

relative  to  the  task  of  isolating  the  fault  in  a  given  item  of  equipment, 
given  that  a  failure  has  been  detected.  The  troubleshoot  data  include 
task  time,  personnel  and  test  equipment  requirements  and  measures 
related  to  the  probability  of  successful  fault  isolation.  Data  are 
entered  for  each  item  of  equipment  designated  on  card  type  31. 

SUBYS.  NO.  The  subsystem  number. 

EQUIP.  NO.  The  equipment  number. 

DIV.  RESP.  The  division  which  contains  the  personnel 

responsible  for  maintaining  the  equipment. 
If  a  blank  is  entered,  no  constraint  on 
division  assignment  is  made. 

TIME  IN  FUNCTION  The  mean  and  standard  deviation  of  the 

MEAN,  STD.  DEV.  time  required  to  isolate  a  malfunction  in 

the  equipment.  A  log  normal  distribution 
is  presently  used  to  randomize  the  actual 
time  using  Monte  Carlo  methods. 

MEN  A  set  of  data  related  to  the  personnel 

required  to  troubleshoot  the  equipment. 

NEC  (IF  REQ.  )  If  special  schooling  is  required  to  qualify 

for  troubleshooting,  the  appropriate  Naval 
Enlisted  Classification  is  entered  (NEC). 

If  called  out  on  card  type  21,  equivalent 
NEC's  may  be  assigned  if  the  original  NEC 
is  not  available,  and  upward  absorption  is 
specified. 
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NO,  RATING,  RATE 


Up  to  four  different  ratings  can  be  used  in 
a  troubleshooting  team.  NO  refers  to  the 
number  of  each  rating  and  RATING  and 
RATE  are  self  explanatory. 

NO.  UP.  In  the  event  the  specified  skill  is  unavailable, 

the  model  is  designed  to  assign  work  to 
higher  skill  levels.  NO.  UP.  is  the  number 
of  rates  higher  than  the  one  specified  to 
which  the  job  can  be  assigned.  If  the  number 
is  greater  than  the  number  of  higher  rates, 
the  search  will  end  with  the  highest  rate. 

EQUIPMENT  (TOOLS)  Up  to  three  different  types  of  test  equipment 

or  special  tools  can  be  used  on  a  job. 

NO  Refers  to  the  number  of  tool  type  i  (i  =  1,  2 

or  3)  required. 

TYPE  Refers  to  the  type  of  tool  required. 

ERROR  PROBABILITIES  (See  Chapter  III,  Section  2a,  Pages  27-31 

for  a  definition  and  discussion  of  error  prob¬ 
abilities). 

PI  The  probability  that  a  type  I  error  will 

occur  in  Troubleshoot. 

P2  The  probability  that  a  type  II  error  will 

occur  in  Troubleshoot. 

P3  The  probability  that  a  type  III  error  will 

occur  in  Troubleshoot. 
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The  probability  that  a  type  t  error  will 
occur  in  Troubleshoot. 

The  average  duration  of  a  t  error. 
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10.  RE  PAIR /TEST  (CM) 

CARD  TYPE  33  contains  data  relative  to 
the  repair  and  repair  verification  of  each  equipment  item.  The  data 
elements  on  this  card  are  similar  to  the  Troubleshoot  data  elements 
(card  type  32),  except  for  maintenance  error  probabilities.  Also,  a 
special  element,  PTR  PTR  (Probability  that  Troubleshoot  is  Required 
Prior  to  Repair),  allows  the  user  to  designate  the  percentage  of  time 
that  troubleshooting  can  be  skipped. 

SUBSYS.  NO.  Subsystem  number. 

EQUIP  NO.  Equipment  number. 

DIV  RESP.  The  division  which  contains  the  personnel 

responsible  for  the  repair  of  the  equipment. 

If  left  blank  any  qualified  man  aboard  ship 
can  be  assigned. 

The  mean  and  standard  deviation  of  the 
time  to  repair  and  test  the  equipment.  A 
log  normal  distribution  is  presently  used 
to  randomize  this  time  via  Monte  Carlo  methods. 

MEN  See  associated  description  with  card  type 

EQUIPMENT  (TOOLS)  32. 

ERROR  PROBABILITIES 

PD  The  probability  of  a  type  d  error. 

P2  The  probability  of  a  type  II  error. 

PTR  PTR  The  probability  that  troubleshoot  is  required 

prior  to  repair.. 


TIME  IN  FUNCTION 
MEAN  STD.  DEV. 
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11.  PLANNED  MAINTENANCE  (PM) 


CARD  TYPE  34  contains  all 

of  the  data  necessary  to  execute  planned  maintenance  during  a  simu¬ 
lation.  The  data  are  based  on  the  Navy's  Planned  Maintenance  System 
(PMS),  and  more  specifically  the  Maintenance  Index  Pages  of  the  PMS 
system.  Each  entry  on  card  type  34  becomes  a'separate  job  in  the 
simulation  which  is  executed  at  the  specified  frequency. 

Frequency  of  the  planned  maintenance 
action.  Allowable  frequencies  are: 

D  Daily 
W  Weekly 
M  Monthly 
Q  Quarterly 
S  Semi-annually 
A  Annually 

All  PM  actions  with  the  same  frequency 
are  listed  together  beginning  with  Dailys 
and  ending  with  Cyclics. 

A  sequential  numbering  of  each  action 
within  a  given  category  (Daily,  Weekly, 
Monthly,  etc. ), 

Subsystem  number  (does  not  have  to  be  in 
order). 

EQUIP  NO  Equipment  number  (does  not  have  to  be  in 

order). 

DIV.  RESP,  The  division  which  contains  the  ratings 

required  to  perform  the  job.  If  blank,  any 


PM  INDEX  (FOlt  EACH 
CAT. )  (Total  of  all  types 
must  be  less  than 
1000) 

SUBSYS.  NO 


FREQ. 
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qualified  personnel  will  be  assigned  regard¬ 
less  of  division. 

TIME  IN  FUNCTION  The  mean  and  standard  deviation  of  the 

MEAN,  STD.  DEV.  time  required  to  perform  the  job. 

MEN/EQUIPMENT  See  card  type  32  for  a  description  of  these 

(TOOLS)  entries. 

SEA/ PORT  An  index  which  indicates  whether  the  job 

must  be  done  in  port.  If  equal  to  p  the  job 
must  be  done  in  port.  Otherwise  it  may 
be  performed  at  any  time. 

CYCLE  TIME  No  longer  used. 
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12.  TOOLS  INVENTORY 


CARD  TYPE  35  contains  a  list  of  the  tools 
and  test  equipment  and  a  count  of  the  number  of  each  type  available 
aboard  ship.  These  items  can  be  called  for  by  Troubleshoot,  Repair/ 
Test  and  Planned  Maintenance  jobs. 

TYPE  A  sequential  number  of  each  type  of  equip- 

(50)  ment. 

DESCRIPTION  A  sixteen  character  field  containing  a  des¬ 

cription  of  the  tool/test  equipment.  Any 
appropriate  designation  may  be  used. 

NO  AVAIL.  The  number  of  each  type  of  tool/test  equip¬ 

ment  available  aboard  ship. 
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1?.  FACILITIES  MAINTENANCE  (FM)  AND  SUPPORT  AND 
ADMINISTRATIVE  (S/A) 

CARD  TYPE  40  contains  the  data  which  describe  the. 

daily  FM  and  S/A  jobs  required  of  the  ship.  Each  entry  represents  an 

individual  job  which  is  assigned  on  a  daily  basis  (excluding  Sunday  or 

Saturday  in  port). 

JOB  NO  The  sequential  number  of  the  job. 

(105) 

DIV  RESP.  The  division  which  is  responsible  for  the 

work. 

FM  or  S/A  FM  if  the  job  is  Facilities  Maintenance, 

S/A  if  the  job  is  Support/ Administrative. 

DESCRIPTION  A  sixteen  character  field  which  contains  a 

description  of  the  work.  Any  appropriate 
description  may  be  used. 

SKILL,  RATING,  RATE  The  skill  type  and  level  to  which  the  job 

should  be  assigned. 

MAN  HOURS  PER  DAY  The  total  man  hours  per  day  required  to 

complete  the  job. 

%  CARRY  The  amount  of  the  daily  work  assignment 

which  should  be  added  to  the  following 
day's  workload  if  not  completed. 
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14.  TRAINING  EXERCISE  REQUIREMENTS 


CARD  TYPE  50  contains 

the  data  necessary  to  schedule  and  perform  required  training  exercises. 
The  input  data  are  based  on  the  requirement  placed  on  destroyer  type 
ships  to  maintain  training  readiness  through  periodic  refresher  exer¬ 
cises.  The  data  include  the  frequency  of  each  of  the  exercises,  identi¬ 
fication  of  special  watch  augmentations,  and  the  mission  functions 
which  must  be  operational  to  perform  the  exercise.  Exercises  are 
delayed  if  any  of  the  specified  mission  functions  are  not  combat  ready. 
Information  on  the  duration  of  an  exercise  and  the  period  during  which 
rescheduling  may  be  attempted  is  contained  in  the  Employment  Schedule 
(card  type  90). 

EXER  NO.  The  sequential  number  of  the  exercise. 

(50) 

DESCRIPTION  A  twenty-four  character  field  which  con¬ 

tains  a  description  of  the  exercise.  Any 
appropriate  designation  may  be  used. 

WHEN  DONE  Some  exercises  cannot  be  performed  at 

night.  A  9  in  this  column  indicates  day¬ 
light  required.  A  1  indicates  that  the 
exercise  may  be  performed  at  any  time. 

COND.  FOR  MANNING  The  basic  readiness  condition  (I,  III,  IV 

or  V)  required  to  perform  the  exercise: 

1  =  I 

2  =  III 

3  =  IV 

4  =  V 

AUG  If  the  basic  readiness  condition  is  to  be 

augmented  such  as  I-AAW,  the  number 
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of  the  augmented  condition  is  entered  in 
AUG.  Up  to  five  different  augmentations 
are  possible  and  these  are  defined  by  the 
user. 


REQUIRED  MISSION  Any  one  or  all  of  up  to  twenty  mission 

FUNCTIONS  functions  may  be  required  to  perform  the 

exercise.  A  1  in  the  appropriate  column 
would  indicate  that  the  mission  function  is 
required. 

| 

I 

i 

I 

J 

t 1 


A- 51 


15.  TOTAL  SHIP  READINESS  SCORING 
CARD  TYPE  60  contains 

the  information  used  to  establish  the  overall  readiness  rating  of  the 
ship.  Four  ship  readiness  states  Cl  (Fully  Combat  Ready),  C2  (Combat 
Ready,  C3  (Marginally  Combat  Ready)  and  C4  (Not  Combat  Ready)  are 
defined  in  terms  of  the  readiness  states  of  the  basic  mission  functions 
(subsystems).  This  information  is  used  during  the  program  to  contin¬ 
uously  evaluate  the  readiness  of  the  total  ship,  which  can  be  affected 
any  time  the  readiness  of  a  mission  function  changes. 

COND  NO.  The  sequential  number  of  the  readiness 

(500  including  card  scoring  condition, 
type  61) 

READINESS  RATING  The  readiness  rating  which  corresponds  to 

the  states  of  the  mission  functions. 

REQUIRED  MINIMUM  Each  of  the  subsystems  defined  as  mission 

CONDITION  OF  THIS  function,  can  be  in  any  of  the  four  readiness 

SUBSYSTEM  FOR  THIS  states.  While  it  is  possible  to  assume  that 
RATING  the  readiness  of  the  ship  is  equal  to  the 

lowest  readiness  of  any  function,  alternative 
interpretations  are  possible.  Therefore 
the  user  enters  in  these  columns  the  com¬ 
binations  of  readiness  states  which  corres¬ 
pond  to  the  overall  ship  readiness  rating. 
The  indicator  D/C  (Don't  Care)  is  provided 
for  those  instances  where  the  loss  of  one 
mission  function  (C4)  may  preclude  the  use 
of  others  and  therefore  their  state  is  of  no 
interest. 


16.  MISSION  FUNCTION  READINESS  SCORING 


CARD  TYPE  61 

contains  the  information  necessary  to  establish  the  readiness  scoring 
of  the  individual  mission-oriented  functions  (also  referred  to  as  sub¬ 
systems).  The  readiness  score  of  a  mission  function  is  completely 
determined  by  the  user  by  identifying  the  readiness  score  (Cl,  C2,  C3, 
or  C4)  which  is  associated  with  a  given  combination  of  up/down  states 
of  the  equipment  which  comprise  the  mission  function. 

SUBSYS.  NO.  The  number  of  the  subsystem  (mission 

function)  which  is  being  scored.  This 
number  must  correspond  with  the  number¬ 
ing  scheme  used  in  card  type  30. 

NO.  EQUIP.  ITEMS  The  number  of  equipment  items  in  the 

subsystem  which  will  be  used  to  score  the 
function. 

COND  NO. 

.  (500  including  card 
type  60) 

'  READINESS  RATING 


REQUIRED  CONDITION  Three  states  may  be  defined  for  each  equip- 

OF  THIS  EQUIPMENT  ment  G  (Good),  B  (Bad)  or  X  (Don't  Care). 

FOR  THIS  RATING  Each  equipment  item  must  have  one  of 

these  three  designations. 

NOTE:  While  the  number  of  possible  up/down  state  combinations 

is  very  large,  the  number  of  probable  combinations,  based 
on  equipment  failure  rate,  is  significantly  less.  For  pur¬ 
poses  of  the  present  effort,  the  expected  availability  of 
equipment  was  used  to  reduce  the  entries  in  the  decisior 
table  to  a  reasonable  nurr.ber 
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A  sequential  listing  of  the  readiness  score 
entries.  Numbering  starts  over  with  each 
subsystem. 

4 

The  readiness  score  of  the  subsystem 
which  corresponds  to  the  equipment  states 
defined  to  the  right. 
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17.  EMPLOYMENT  SCHEDULE 

CARD  TYPE  90  contains  a  sequential 
listing  of  commands  which  constitute  the  mission  scenario.  Special 
commands  are  also  provided  for  ordering  output  reports  during  the 
simulation.  Each  command  is  associated  with  a  time  of  execution  which 
corresponds  with  the  time  of  simulation. 

DEPART  Puts  ship  to  sea  at  condition  IV  steaming. 

Establishes  mission  termination  time  and 
assigns  name  to  mission  for  all  output. 

Puts  ship  into  port  at  designated  time  and 
stays  there  until  SEA  command  causes  it 
to  leave. 

Takes  ship  out  of  port  at  designated  time. 
While  ship  is  at  sea,  training  exercises 
can  be  assigned,  readiness  conditions  can 
be  imposed,  and  evolution  can  occur. 

Causes  the  identified  training  exercise  to 
be  assigned  within  the  designated  period 
for  the  specified  duration.  Its  performance 
is  a  function  of  the  required  equipment 
state  which  may  cause  delay. 

EVOLUTE  NO  Causes  the  identified  evolution  to  be 

(500  including  assigned  for  the  designated  period. 

"TRAINEX") 

ACTION  POSSIBLE 
(100  including 
"ACTION 
IMMINENT") 

P*  -  5  7 


Imposes  readiness  condition  III  on  the  ship 
for  the  specified  duration  while  the  ship  is 
at  sea.  When  the  period  ends,  the  ship 
returns  to  condition  IV  steaming. 


TRAINEX  NO 

(500  including 
"EVOLUTE") 


SEA 

(50) 


PORT 

(50) 


ACTION  IMMINENT  Imposes  readiness  condition  I  on  the  ship 

(100  including  for  the  specified  duration  while  the  ship  is 

"ACTION  at  sea.  When  the  period  ends,  the  ship 

POSSIBLE")  returns  to  condition  IV  steaming. 

PRINT  OUTPUT  Causes  a  full  set  of  output  reports  to  be 

printed  starting  at  the  designated  time  and 
continuing  to  print  another  full  set  at  the 
specified  interval. 

PRINT  SNAPSHOT  Causes  a  snapshot  to  be  printed  at  the 

designated  interval,  starting  at  the  specified 
time. 

ANCHORS  AWEIGIi  Denotes  the  end  of  all  the  data  and  the  start 

of  computation. 
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APPENDIX  B 


SHIP  II  OUTPUT  REPORTS 


A.  Description  of  Heading  Information 

Each  of  the  nine  types  of  reports  to  be  described  contain 
an  identical  heading.  The  heading  contains  information  about 
the  study  being  run  and  the  time  at  which  the  report  v/as 
generated.  The  mission  name  is  an  identifying  term  input 
to  the  model  by  the  user.  Day  NO  identifies  the  time  in  days 
within  the  simulation  run  at  which  the  report  was  output. 
Normally  Ship  II  output  reports  are  generated  every  seven 
days.  Subseauent  reports  would  therefore  be  generated  on 
days  14,  21,  28,  etc.  Time  (hrs)  also  identifies  the  time 
within  the  simulation  run  at  which  the  report  was  generated, 
except  that  time  is  presented  in  hours  rather  than  days. 

Time  of  day  represents  the  time  of  day  at  which  the  report 
was  generated.  The  output  report  examples  given  later  con¬ 
tain  a  value  of  zero  for  time  of  day.  This  value  represents 
time  0000  on  a  24  hour  clock.  Day  of  week  is  simply  the 
day  of  week  at  which  the  report  was  generated.  The  report 
interval,  day  of  week  and  time  of  day  are  determined  by  the 
user  and  input  to  the  model  as  part  of  the  Employment  Schedule. 
The  last  item  of  the  heading  is  the  division  number  and  name, 
both  of  which  are  specified  by  the  user  and  are  part  of  the 
input  data. 


B.  Discussion  of  Output  Reports 

The  following  sections  contain  a  brief  explanation  of 
each  SHIP  II  Output  report.  An  example  of  the  output  report 
follows  each  discussion. 

1.  Personnel  Record 


This  report  summarizes  the  workload  for  each  crewman. 
The  first  seven  columns  are  identifying  information  which  is 
taken  from  model  input  data.  The  information  in  the  following 
ten  columns  represents  hours  worked  in  each  of  nine  task 
categories  and  total  hours  worked.  The  last  column  (labeled 
NWW)  is  the  percent  of  standard  Navy  work  week  which  is 
based  on  the  total  hours  worked  for  each  man.  The  model 
calculates  this  percentage  based  on  a  74  hour  workweek  for 
watchstanders  and  a  £6  hour  workweek  for  non-watchstanders . 


The  information  in  each  column  is  totaled  for  the 


£ 


t 

f 


total  divisions  and  for  watchstanders  and  non-watchstanders 
separately.  The  personnel  record  also  accumulates  data  for 
the  total  ship  (not  shown) ,  when  more  than  one  division  is 
used.  In  this  case  a  separate  page  is  alloted  to  each 
division. 

Abbreviations ; 

Watch  -  Watch  hours 

RTRA  -  Readiness  training 

EVOL  -  Evolutions 

PM  -  Planned  maintenance 

CM  -  Corrective  maintenance 

FM  -  Facilities  maintenance 

S/A  -  Support  and  administrative 

SD  -  Service  diversions 

NWW  -  Percent  of  standard  Navy  work  week. 
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This  report;  summarizes  the  performance  of  each  type 
of  maintenance  personnel  summarized  by  rate.  The  first  two 
columns  identify  rate  and  rating.  The  column  labeled  "CAT" 
contains  rating  information  coded  as  follows: 

1  =  Chief 

2  =  1st  class 

3  <=  2nd  class 


4  = 

3rd 

class 

5  = 

FN, 

SN, 

etc . 

6  = 
7  = 

XN 

FA, 

SA, 

etc . 

The  third  column  contains  the  number  of  each  rate/ 
rating  combination  used  in  the  simulation. 

The  next  two  columns  ("CM  hrs/W"  and  "PM  hrs/W")  con¬ 
tain  the  total  CM  and  PM  hours  for  each  class  of  personnel. 
The  "total"  column  contains  the  sum  of  the  PM  and  CM  columns. 
The  column  labeled  "NO  QS"  contains  the  number  of  times  that 
a  maintenance  job  was  delayed  while  waiting  for  maintenance 
personnel  of  each  rate/rating  to  become  available  for  the 
job.  The  column  labeled  "Q  HRS"  contains  the  total  time  that 
corrective  maintenance  jobs  were  waiting  for  personnel  to 
become  available. 

The  column  "NO  INRPT"  contains  the  number  of  times 
that  planned  and  corrective  maintenance  jobs  were  interrupted 
to  get  a  man  of  the  rate/rating  specified  in  columns  1  and  2. 

Abbreviations : 

CAT  =  category 

NOB  -  number  on  board 

CM  HRS/W  =  corrective  maintenance  hours/week 

PM  HRS/W  =  planned  maintenance  hours/week 

NO  QS  =  number  of  queues 

Q  HRS  =  queue  hours  * 

NO  INRPT  =  number  of  interrupts 
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3 .  Maintenance  Personnel  Summary-NEC 

lliis  report  is  identical  to  the  preceding  report 
except  that  personnel  are  specified  by  NEC. 
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4 .  Equipment  Summary 


The  first  three  columns  contain  identifying  infor¬ 
mation  from  model  input  data;  the  first  two  specify  the 
subsystem  number  and  equipment  number  within  the  subsystem. 
The  equipment  name  is  in  the  third  column.  The  next  column 
"OP  T"  specifies  the  total  number  of  hours  (since  the  last 
set  of  output  reports)  for  which  the  equipment  was  operating 
properly.  "DOWN  T"  contains  the  total  hours  during  which 
the  equipment  was  malfunctioning.  The  next  column  contains 
the  availability  percent  -  the  percentage  of  the  time  since 
the  last  output  that  the  equipment  was  operating  properly. 

The  number  of  equipment  failures  is  listed  in  the 
next  column,  followed  by  the  total  time  for  detection  of 
failures.  The  column  labeled  "Defers"  contains  the  total 
time  for  which  equipment  waited  to  be  repaired  because  of 
deferrals  for  parts  and/or  assistance. 

The  next  four  columns  contain  data  on  corrective 
maintenance  performance  for  each  eauipment.  The  columns 
labeled  "TS  Q"  and  "REP  Q"  contain  the  total  number  of 
queue  hours  for  troubleshooting  and  repair,  respectively. 
These  queues  are  caused  by  unavailability  of  personnel, 
parts,  or  equipment.  The  columns  labeled  "TS  TIF"  and 
"REP  TIF"  contain  the  total  time  spent  in  the  troubleshooting 
and  repair  functions  for  each  equipment. 

The  next  column  contains  the  actual  and  apparent 
status  of  equipment.  The  right  hand  half  of  the  column 
contains  the  apparent  current ■ status  of  the  equipment  and 
the  left  hand  side  contains  the  actual  equipment  status. 
Usually  actual  and  apparent  status  will  be  the  same. 

However,  when  certain  types  of  troubleshooting  and  repair 
errors  (see  Section  III-B)  are  simulated,  actual  and  ap¬ 
parent  status  will  differ  after  the  error  has  occurred. 

These  errors  usually  result  in  good  apparent  condition  and 
bad  actual  condition. 

The  column  labeled  "TECS"  contains  the  time  in  • 
hours  from  the  start  of  the  simulation  when  the  equipment 
entered  the  condition  shown  in  the  status  column. 

Abbreviations ; 

SS  =  Subsystem  number 

EQP  =  Equipment  number 

OP  T  r  Operating  time 

DOWN  T  =  Down  time  (time  during  which  the  equipment 
was  not  operating) 

AVL  PCT  =  Availability  percent 
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FAIL  =  Number  of  failures 

DET  T  =  Detection  time 

DEFERS  =  Deferral  hours 

TS  Q  =  Troubleshooting  queues  hours 

TS  TIF  =  Troubleshooting  time  in  function 

REP  Q  =  Repair  queue  hours 

REP  TIF  =  Repair  time  in  function 

TECS  =  Time  entered  current  function 

The  abbreviations  used  in  equipment  names  are  standard 
Navy  abbreviations. 
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5 .  Planned  Maintenance  Summary 

A  separate  planned  maintenance  summary  is  output  for 
each  PM  task  frequency  (daily,  weekly,  monthly,  quarterly, 
semi-annual  and  annual) .  An  example  of  a  report  of  each 
type  is  included.  Note  that  the  format  is  identical  for 
each  type  of  report. 

The  first  two  columns  contain  subsystem  and  equipment 
numbers  taken  from  input  data.  The  third  column  contains 
the  PM  task  number.  Each  PM  task  is  given  a  separate  task 
number  by  the  user  and  compirses  part  of  the  PM  input  data. 

The  fourth  column  specifies  the  tasK  frequency,  using 
the  following  code:  D  =  daily,  W  =  weekly,  M  =  monthly, 

Q  =  quarterly,  S  =  serai -annual ,  and  A  =  annual.  This  infor¬ 
mation  is  also  taken  from  input  data. 

The  information  in  the  next  five  columns  is  generated 
by  the  simulation.  The  column  labeled  "sched"  contains  the 
number  of  times  the  PM  task  was  scheduled.  The  next  two 
columns  tell  the  frequency  with  which  the  task  wa<=  delayed 
and  cancelled.  The  column  labeled  "FAILS  FND"  contains  the 
number  of  times  that  an  equipment  failure  was  found  during 
PM.  The  last  column  contains  the  total  manhours  spent  on 
each  PM  task. 

A  sum  of  the  entries  in  the  last  five  columns  for  all 
PM  tasks  of  each  type  is  given  on  the  last  line  of  each 
report.  The  line  containing  these  sums  is  identified  by 
a  zero  in  the  "FREQ"  column.  . 

Abbreviations : 


SS  =  Subsystem  number 

EQP  =  Eouipment  number 

NO  =  PM  task  number 

FREQ  =  Interval  of  performance 

SCHED  =  Number  of  times  the  task  was  scheduled 

DELAYED  =  Number  of  times  the  task  was  delayed  • 

CANCELLED  =  Number  of  times  the  task  was  cancelled 

FAILS  FND  =  Failures  found 

TOT  MHRS  =  Total  manhours 
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6 .  Facilities  Maintenance  Summary 

The  first  four  columns  contain  identifying  information 
taken  f^om  input  data.  The  first  column  contains  the  indi¬ 
vidual  job  number;  the  second  column  contains  the  division 
assignment;  only  personnel  from  the  assigned  division  will 
be  used  fcr  FM  tasks.  A  zero  in  the  division  column  indicates 
a  job  not  assigned  to  any  particular  division. ^  In  the  third 
column  a  description  of  the  job  can  be  included  if  desired. 

The  fourth  column  identifies  the  job  as  either  FM  or  S/A. 

The  last  five  columns  contain  simulation  output.  The 
first  of  these,  "MHRS/D"  tells  how  many  manhours  per  day  were 
scheduled  for  each  job.  The  next  column  contains  the  number 
of  times  that  each  job  was  scheduled  in  the  reporting  period. 
The  column  labeled  "HRS  LEFT"  contains  the  total  number  of 
man  hours  scheduled  but  not  done.  The  following  column  con¬ 
tains  the  number  of  days  on  which  the  job  was  scheduled  but 
not  accomplished.  The  information  in  the  last  column  tells 
the  total  number  of  times  a  job  was  started  but  interrupted 
by  a  higher  priority  task  before  it  could  be  completed. 

After  each  job  has  been  listed,  a  summary  by  division 
and  total  ship  is  provided.  The  first  line  which  has  no 
entry  in  the  "JOB"  column  is  the  start  of  this  summary.  This 
line  will  have  a  "0"  in  the  division  column  and  contains  a 
total  for  all  jobs  not  assigned  a  specific  division 
responsibility.  Following  this  will  be  a  total  for  each 
division  listed  by  division  number.  The  last  line  will  have 
"99"  in  the  division  column,  which  identifies  the  total  ship 
summary  of  data. 

Abbreviations : 

JOB  =  FM-S/A  job  number 

DIV  =  Division 

MHRS/D  =  Man  hours  scheduled  per  day 

NO  SCHED  =  Number  scheduled 

HRS  LEFT  =  Hours  left 


iThe  output  report  example  was  taken  from  a  study  using 
only  one  division. 
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7.  Division  Training  Summar 


The  first  two  columns  of  this  summary  contain  the 
division  number  and  name.  The  third  column  contains  the 
minimum  attendance  necessary  for  a  class  to  be  held;  this 
value  is  taken  from  input  data.  The  next  column  contains 
the  total  number  of  different  men  who  have  attended  a  division 
training  class.  The  column  labeled  "total  hrs"  contains 
the  total  number  of  class  hours  (not  man  hours  in  class) . 

The  last  three  columns  contain  the  total  number  of  class 
sessions  which  occurred,  the  average  attendance,  and  the 
average  class  duration. 
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8 .  Training  Readiness  Summary 

The  first  two  columns  contain  the  number  and  name  of 
each  training  exercise.  The  next  four  columns  contain  the 
number  of  times  each  exercise  was  scheduled,  performed,  can¬ 
celled  and  delayed.  The  last  two  columns  contain  number  of 
delays  and  total  delay  hours  for  those  delays  caused  by  a 
required  equipment  being  unavailable. 

The  last  row  on  the  report  (identified  by  a  zero  in 
the  "NO"  column)  contains  a  total  of  the  entries  in  the  last 
six  columns. 


Abbreviations : 

SS  DELAYS  =  Subsystem  delays 
DELAY  HRS  =  Delay  hours 


NAVAL  PERSaNMFt  RESEARCH  AND  IiE VEU0PHENT  LAR0RAT0RY 

•  •  •  M  1  S  S 1  3  M  NaM£*  STG  STUDY  RUN  1  6  MfcN  •  •  • 

•  •  •  TlMfc  (HHS)«  168,02  DAY  NR*  7  •  •  • 

•  •  •  time  3*day*  o  day  9*week»  mrn  *  •  • 

•  •  •  TRAINING  readiness  summary  •  •  • 


9.  Readiness  Summar 


This  report  summarizes  the  readiness  condition  of  all 
the  subsystems  listed  in  column  two  of  the  report.  The  only 
subsystems  which  are  listed  in  column  two  are  those  designated 
by  the  user  as  subsystems  whose  correct  performance  is  nec¬ 
essary  for  the  total  ship  to  accomplish  its  mission. 

The  main  body  of  the  table  shows  for  each  subsystem 
the  number  of  times  that  the  subsystem  was  in  each  readiness 
status,  the  total  hrurs  in  each  status,  the  average  hours  in 
each  status  and  the  percent  of  total  time  in  each  status. 

The  abbreviation?  C-l,  C-2,  C-3,  and  C-4  represent  the 
standard  Navy  readiness  status  designations.  The  column 
labeled  "current"  contains  the  readiness  status  of  each  sub¬ 
system  at  the  time  of  the  report.  The  last  column  contains 
the  time  at  which  the  current  status  was  entered,  measured 
in  hours  from  the  start  of  the  simulation  run. 

After  all  subsystems  have  been  listed,  the  report 
summarizes  the  readiness  status  of  the  total  ship.  The 
relationship  between  subsystem  readiness  and  total  ship 
readiness  is  specified  by  the  user.  However,  Ship  II  cur¬ 
rently  uses  only  subsystem  state  of  mission-related  subsystems 
to  calculate  total  ship  readiness  status. 

Abbreviations : 

TECS  =  Time  entered  current  function 
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